Squid Cache, TProxy, dan Mikrotik (Alternate Configuration for Simple Networks)

This is the english version of this indonesian article;

IMO, there 2 types of network topology involving linux machine with squid installed doing cache on simple network (with mikrotik os spesific router). This article tends to give alternative topology using tproxy (A set of kernel patches reimplementing support for userspace transparent proxies).

Those 2 types are;

1. Squid is in equal position with the client, the http port interception is done through dst-nat or ip proxy parent with redirect port configuration (Configuration A).
2. Squid is after the client’s router and before the internet, the http port intercaption done through iptables redirect port (or transparent).

Both of the models have significant downside as follows (subjective opinion);

Only one IP Address recognized after Squid because of NAT which caused;

1. For networks counting on Mikrotik Router OS as bandwidth shaper, request to the internet coming from Squid cannot be shaped enymore (per client basis), which also automatically will eat all available bandwith, and another shape will be done globally by Internet Service Provider (Bandwidth Provider) in their router.
2. Bandwitdh coming out from Squid to the users (although taken from cache) will be included into client’s shape and limit (not a fair solution if we provide a dedicated bandwitdh for our client).
3. For multiple Gateway Connection (Configured by the Router – eg. Internasional through ADSL and wireless configuration), Squid will only use one gateway (the default gateway) – not a problem if International through ADSL and IIX through Wireless. It will be a problem if there is a load balancing configuration (client’s group or sessions based).

Intention

Building Squid in bridged configuration without any additional configuration, not possible at this moment, since Squid works in application layer (layer 3) and required routing. Guide on http://freshmeat.net/articles/view/1433/ not able to give solution to the downside mentioned above, because after Squid there will be only one IP Address. Although it is possible to implement the configuration in simple internet cafe without plenty of subnets. You could find another useful guide on bridged tpoxy through this article TProxy pada bridge;

Alternative Description

Squid become transition gateway between the clients and router (Gateway to the clients), the linux machine will route internet request package to the router without doing NAT by using Tproxy (patching linux kernel and iptables) and tcp_outgoing_address directive in squid.conf. The requiest on http port (80) will be routed to to the upper router (mikrotik in this case). Then static routing entries have to be configured in the mikrotik router which will route the answer from the internet to the Squid TProxy.

Data flow (HTTP) become.

Client (172.16.17.4) –> eth1(172.16.17.1) (port 80 interception) –> port 3128 (squid) witdh tcp_outgoing_address menggunakan 172.16.17.1 (client gateway) —-> routing (without NAT) –> Mikrotik (shaping + NAT) –> Another Router–> Internet –> Back to Mikrotik –> Static Routing to Squid Gateway–> Port 80 interception –> Client

These are the general steps;

1. Standard Debian Etch Installation, Kernel 2.6.18
2. cttproxy kernel patch
3. TPROXY iptables userspace patch
4. Squid installation from source
5. Squid configuration
6. Routing configuration (without NAT) with Firehol
7. Static Routing in Mikrotik Router

Disclaimer

1. Tested in my own network.
2. This article is not intended to describe squid optimalization.
3. Possibility to be an incorrect solution (this is experimental)
4. Most importantly, I am a Linux and Networking N00bs.

More Detailed Steps;

1. Standard Debian Etch Instalation, Kernel 2.6.18-4

Base install, fetch Debian Etch net-install ISO from your favorite mirror, arrange the HDD partition scheme to your flavour. Mine was;

df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 14G 1.2G 12G 9% /
tmpfs 253M 0 253M 0% /lib/init/rw
udev 10M 48K 10M 1% /dev
tmpfs 253M 0 253M 0% /dev/shm
/dev/sda2 19G 278M 19G 2% /cache1
/dev/sda3 19G 278M 19G 2% /cache2
/dev/sda6 24G 223M 22G 1% /home

2. cttproxy Kernel Patching

Read more information about kernel patching and compilation on howtoforge, I use the first method (Building a kernel .deb package), which steps are as follows (re-read the guide on howtoforge if you failed to follow my steps);

Required package installation (for kernel compilation);

apt-get update
apt-get install kernel-package libncurses5-dev fakeroot wget bzip2 build-essential

Download source kernel, the same kernel in this case since compilation will be done not for upgrading purpose;

cd /usr/src/
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.18.4.tar.gz
tar -xzvf linux-2.6.18.4.tar.gz
ln -s linux-2.6.18.4 linux

Download and Apply kernel patch from cttproxy, read the detailed README in the package.

cd /usr/src/
wget http://www.balabit.com/downloads/files/tproxy/obsolete/linux-2.6/cttproxy-2.6.18-2.0.6.tar.gz
cd /usr/src/linux
for i in <pathtocttproxy>/patch_tree/0{1,2,3}*.diff; do cat $i | patch -p1; done

Compile the kernel, activate conntrack, NAT and TPROXY support (inside Netfilter Configuration)

make clean && make mrproper
cp /boot/config-`uname -r` ./.config

Inside Kernel Compilation menu, choose Load an Alternate Configuration File and choose .config

menuconfig position

-> Networking
---> Networking support
-----> Networking options
-------> Network packet filtering
----------> IP: Netfilter Configuration

After done configuring, exit the menuconfig, answer Yes on question Do you wish to save your new kernel configuration?

Build the Kernel

make-kpkg clean
fakeroot make-kpkg --initrd --append-to-version=-custom kernel_image kernel_headers

Wait.., Install the newly compiled kernel

cd /usr/src/
ls -l<

thesw 2 files should exist;

linux-headers-2.6.18.4-custom_2.6.18.4-custom-10.00.Custom_i386.deb
linux-image-2.6.18.4-custom_2.6.18.4-custom-10.00.Custom_i386.deb

Install those .deb;

dpkg -i linux-headers-2.6.18.4-custom_2.6.18.4-custom-10.00.Custom_i386.deb
dpkg -i linux-image-2.6.18.4-custom_2.6.18.4-custom-10.00.Custom_i386.deb

GRUB bootloader will be done automatically applying the new kernel, restart the system.

shutdown -r now

3. TPROXY iptables userspace patching

Fetch iptables source

cd /usr/src/
apt-get source iptables

patch iptables (detailed info in README cttproxy)

cd /usr/src/iptables-1.3.6.0debian1
cat <pathtocttproxy>/iptables/iptables-1.3-cttproxy.diff | patch -p1
chmod +x extensions/.tproxy-test
make KERNELDIR=/usr/src/linux

Build the .deb, and install

cd /usr/src/iptables-1.3.6.0debian1
dpkg-buildpackage -b -rfakeroot
cd /usr/src/
dpkg -i iptables_1.3.6.0debian1-5_i386.deb

Create TPROXY rule to intercept HTTP request packets

iptables -t tproxy -A PREROUTING -i eth1 -p tcp --dport 80 -j TPROXY --on-port 3128

In Debian, put this command inside /etc/rc.local to automatically applied after restart, assumption = clients connected through eth1

4. Install Squid from the source (Squid Binary 2.6.STABLE14)

Fetch directly from squid-cache website, since squid package from debian doesn’t support tproxy;

http://packages.debian.org/changelogs/pool/main/s/squid/squid_2.6.5-6/changelog

squid (2.6.5-2) unstable; urgency=low
* debian/rules
- Remove mispelled configure option enablig TPROXY support
(TPROXY support is NOT enabled since it needs kernel patches which
are not in the kernel sources distributed by debian)

cd /usr/src/
wget http://www.squid-cache.org/Versions/v2/2.6/squid-2.6.STABLE14.tar.gz
tar -xzvf squid-2.6.STABLE14.tar.gz

Compile Squid, same configuration option with standard installation, witch additional option –enable-linux-tproxy, if you wish to view the standard configuration of your Squid, install squid with apt-get and run squid witch -v

apt-get install squid
squid -v

Install Squid from the source, run .configure (in one line)

cd /usr/src/squid-2.6.STABLE14/
./configure --prefix=/usr --exec_prefix=/usr --bindir=/usr/sbin --sbindir=/usr/sbin --libexecdir=/usr/lib/squid --sysconfdir=/etc/squid --localstatedir=/var/spool/squid --datadir=/usr/share/squid --enable-linux-netfilter --enable-storeio=ufs,aufs,diskd,null --enable-arp-acl --enable-removal-policies=lru,heap --enable-snmp --enable-delay-pools --enable-htcp --enable-poll --enable-cache-digests --enable-underscores --enable-referer-log --enable-useragent-log --enable-auth="basic,digest,ntlm" --enable-carp --enable-large-files --enable-linux-tproxy
make all
make install
cp /usr/src/linux/include/linux/netfilter_ipv4/ip_tproxy.h /usr/include/linux/netfilter_ipv4
cp /usr/include/linux/capability.h /usr/include/sys

5. Squid Configuration

Principally, Squid required routing to route the internet request, routing is depends on your linux routing table;

route
Destination Gateway Genmask Flags Metric Ref Use Iface
10.40.40.0 * 255.255.255.252 U 0 0 0 eth0
172.16.80.0 * 255.255.255.248 U 0 0 0 eth1
default 10.40.40.1 0.0.0.0 UG 0 0 0 eth0

The default gateway is 10.40.40.1, and by default, all package from Squid will route to this IP Address. What we want to do here is to keep the Local IP Address or subnet (client’s subnet) when routed to the upper router by Squid. This will be done by binding the client’s real IP Address to the Linux Local IP Address (client’s gateway).
On Squid, after we activate the tproxy, and put tcp_outgoing_address based on src-address, these are part of my configuration, this configuration might differ for recent Squid (3.0++)

http_port 3128 tproxy transparent

##ACLs###
acl client_3dnet_isplasa src 172.16.17.0/255.255.255.248
acl client_3dnet_alwy src 172.16.17.8/255.255.255.248
acl client_3dnet_anis src 172.16.17.16/255.255.255.248
acl client_3dnet_pkstebet src 172.16.17.24/255.255.255.248
acl client_3dnet_pkstebet2 src 172.16.17.32/255.255.255.248
acl client_sonny src 172.16.35.8/255.255.255.248
acl client_christ src 172.16.35.16/255.255.255.248
acl client_shandy src 172.16.35.24/255.255.255.248
acl client_tono src 172.16.35.32/255.255.255.248
…

##TCP_Outgoing_Address###
tcp_outgoing_address 172.16.17.1 client_3dnet_isplasa
tcp_outgoing_address 172.16.17.9 client_3dnet_alwy
tcp_outgoing_address 172.16.17.17 client_3dnet_anis
tcp_outgoing_address 172.16.17.25 client_3dnet_pkstebet
tcp_outgoing_address 172.16.17.33 client_3dnet_pkstebet2
tcp_outgoing_address 172.16.35.9 client_sonny
tcp_outgoing_address 172.16.35.17 client_christ
tcp_outgoing_address 172.16.35.25 client_shandy
tcp_outgoing_address 172.16.35.33 client_tono
…

##Default##
tcp_outgoing_address 10.40.40.2
server_persistent_connections off

With the configuration, hopefully when a request come from 172.16.17.4, squid on 172.16.17.1:3128 will route the request with local client;s IP Address (172.16.71.4) to Mikrotik. Please make sure that 172.16.17.1 (Client’s Gateway) is assigned to the Linux Local Interface.

6. NAT Without Routing with Firehol

Normally an internet gateway (on a non-public IP Address configuration) will NAT the request from their local users before route the traffic to the upper router, this is what we don’t one.

For this requirement, I use Firehol, you can read all about it on http://firehol.sourceforge.net, this my configuration sample;

Install Firehol
apt-get install firehol

For Firehol applied during reboot

nano /etc/default/firehol
#Rubah NO menjadi YES
START_FIREHOL=YES
#If you want to have firehol wait for an iface to be up add it here
WAIT_FOR_IFACE=”"

Standard Configuration

cat /etc/firehol/firehol.conf

version 5

interface eth0 tomt
policy accept

interface eth1 toclient
policy accept

## for eth1 could route to eth0 ##
router localpublic inface eth1 outface eth0
route all accept

## for eth0 could route to eth1 ##
router publiclocal inface eth0 outface eth1
route all accept

## for local packets##
router locallocal inface eth1 outface eth1
route all accept

Run Firehol

/etc/init.d/firehol start

Please notice that after you run firehol, all the iptables rule applied previously will be flushed. So re-applied the iptables TPROXY after you start or restart Firehol.

7. Static Routing in Mikrotik Router

For Mikrotik router to understand how to reach the client’s and answer their request, static routing must be applied here;

/ip route pr
..
A S 172.16.80.0/24 r 10.40.40.2 local
..

Create the rule for each of your local client’s subnet.

Additionally, if you want your Linux Box also become the DNS for the client’s, install bind, and make sure you have valid nameserver in /etc/resolv.conf.

apt-get install bind
cat /etc/resolv.conf
nameserver 10.40.40.1

special thanks to;

Logan, Rizal dari LintasWave
Baba, Maulana
especially Google