Introduction

Previously, all the hosts on my network would use auto proxy config (DHCP or DNS), but that is insecure and requires additional steps to configure on every host. Also, Android (as of 4.4.4) does not support auto proxy configuration.

Squid Tproxy support is described in quite a bit of detail in Feature: TPROXY version 4.1+ Support document on official Squid web site. However, my setup is slightly more involved due to the fact that my Squid box is not my router. My router is a MikroTik RB750GL - a great little device!

There are some pages on the web that describe somewhat similar setups (e.g. Squid in TProxy mode on Fedora, Mikrotik, squid and tproxy), but none of them had the complete solution. So, here is one.

The topology

There is one LAN segment with multiple hosts on this /24 network. Your typical home setup. One of the machines on this LAN is the Squid box. To get out of this network to the internet, all hosts route through the MikroTik box, which is every host's default gateway. So far, easy.

All hosts on LAN are essentially connected to ether2-master-local interface on the MikroTik router. There is a PPPoE connection going over ether1-gateway out to the internet.

Squid box has just one interface, eth0.

The plan

So, as packets destined for port 80 on web servers located on the internet arrive at MikroTik router, they need to be reflected back to the Squid machine unchanged. The Squid machine then takes the packets, routes them to the IP/port where Squid is listening in Tproxy mode (i.e. locally). Of course, this should only apply to packets that are not coming from the Squid box itself.

Once Squid gets the packets, it will send them off to the internet, through the MikroTik router again. This time the packets will go out (i.e. they will not be reflected back to Squid box), after being masqueraded with the public IP address assigned to the internet connection. Note that in Tproxy mode, Squid will be spoofing client's IP addresses when sending packets to the router.

When the internet web server replies back, the packets will be de-NAT-ed on the MikroTik router and then sent back to LAN. Given that Squid spoofed original client's IP address, absent any special rules on the router, these packets would be routed directly to clients. This is not what is required - all packets need to go back to the Squid box. Once there, Squid will reply back the the client. This way, Squid can cache the pages (and of course, client would not know what to do with these packets anyway - there were in reply to the Squid box, not the client).

The routes are:

Out: client -> mikrotik -> squid -> mikrotik -> internet

In: internet -> mikrotik -> squid -> client

Implementation

Both the MikroTik router and the Squid box require specific configurations to make this work.

MikroTik

On the MikroTik router, I inserted three rules. The first rule marks outgoing packets that need to be reflected back to Squid:

/ip firewall mangle add action=mark-routing chain=prerouting dst-address=!ip-range-on-ether1-gateway dst-port=80 in-interface=ether2-master-local new-routing-mark=tproxy protocol=tcp src-mac-address=!squid-mac-address

The next rule marks incoming packets that need to be sent to Squid:

/ip firewall mangle add action=mark-routing chain=prerouting in-interface=all-ppp new-routing-mark=tproxy protocol=tcp src-port=80

The last rule actually routes the marked packets to the Squid box:

/ip route add distance=1 gateway=squid-ip-address routing-mark=tproxy

Another very important detail in this setup is to disable ICMP redirects being sent and accepted from/on the router:

/ip settings set send-redirects=no

Without this, clients that are set up to accept routing redirects will get confused, because they may get redirected to Squid box for routing. On my MikroTik router, acceptance of redirects was disabled by default.

Squid

On the Squid box, routing needs to be enabled and we also want to disable ICMP redirects. So, I created /etc/sysctl.d/10-tproxy.conf and placed this into the file:

net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.default.forwarding = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
net.ipv4.conf.default.send_redirects = 0

Note that many other examples talk about disabling reverse path filtering (i.e. net.ipv4.conf.all.rp_filter). That may be required for some setups, but not for this one. That is because the Squid box already has a default route through eth0, so all packets eventually go that way anyway.

Next, we need to mark the packets that were reflected from the MikroTik router or that came back from web servers on the internet, so that we can route them to Squid. These packets will be going to port 80 or will match an already held socket and will have destination addresses that are not local to the Squid box. My Squid box also runs Apache on port 80, so I wanted to make sure that those packets do not end up going to Squid. The relevant iptables rules are (in /etc/sysconfig/iptables):

*mangle
:TPXYSCKT - [0:0]
-A PREROUTING -i eth0 -p tcp -m addrtype ! --dst-type LOCAL -m socket --transparent -j TPXYSCKT
-A PREROUTING -i eth0 -p tcp -m addrtype ! --dst-type LOCAL -m tcp --dport 80 -j TPROXY --on-port 3128 --on-ip 127.0.0.1 --tproxy-mark 0x1/0xffffffff
-A TPXYSCKT -j MARK --set-xmark 0x1/0xffffffff
-A TPXYSCKT -j ACCEPT

So, a new chain TPXYSCKT is created. If an already existing socket matches (because Squid has it already), the packets are directed into the chain, marked and accepted. Otherwise, they hit the second rule where they are marked by destination port.

Note that you also have to open port 80 in the INPUT chain, so that packets can come in. And, you should allow forwarding of port 80. For this, I have:

-A INPUT -i eth0 -p tcp -m tcp --dport 80 -j ACCEPT
-A FORWARD -i eth0 -p tcp -m multiport --ports 80 -j ACCEPT

Of course, I run Apache on the box as well, so your INPUT rule may even be more restrictive. Something like:

-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -m addrtype ! --dst-type LOCAL -j ACCEPT

Then, we set up special routing rules for the marked packets (I placed this in /etc/rc.d/rc.local file):

ip rule add fwmark 1 lookup 10
ip -f inet route add local default dev lo table 10

Second to last step, we permanently set all Squid SELinux booleans to true:

setsebool -P squid_use_tproxy=1
setsebool -P squid_connect_any=1

Finally, we configure Tproxy support in Squid. Note how it is sufficient for Squid to listen on the loopback interface for this to work:

http_port 127.0.0.1:3128 tproxy

This will still give you the opportunity to listen using regular Squid port on the interface that clients can directly see (in my case eth0), for explicit or auto configuration.

Updates

If you are trying to do this on Fedora 21, you may want to check out bug #1145235. There are some additional hoops to jump through.

Conclusion

A bit tricky, but can be made to work with a bit of effort. For now (RouterOS 6.21.1), MikroTik routers cannot do a routing mark for IPv6. However, once that becomes available, essentially equivalent config should work for that protocol as well.

Copyright © 2014 Bojan Smojver.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the licence is here.