I am trying to get HAProxy up and running with TPROXY support...I've built
1.5-dev18 with the proper flags, and I'm running on a kernel with the
correct support.
My config is something like this:
global
log 127.0.0.1 local0 notice
maxconn 200
daemon
defaults
option tcplog
log global
mode http
timeout client 5000ms
timeout connect 50000ms
timeout server 50000ms
option contstats
listen www
bind :80
mode tcp
maxconn 50000
source 10.252.168.252 usesrc clientip
server 10.244.46.81
option redispatch
If I comment out the 'source 10.252.168.252 usesrc clientip' line out,
everything works great....but the server on the far end receives the IP of
the haproxy node.
If I leave that 'source' line in, I don't seem to get any responses.
Turning on some packet traces I see the following
*NOTE: my client is 74.112.38.136
<On the HAProxy node, 10.252.168.252>
1832.467324 74.112.38.136 -> 10.252.168.252 TCP 52916 > https [SYN] Seq=0
Win=65535 Len=0 MSS=1460 WS=4 TSV=1260124448 TSER=0
1832.467385 10.252.168.252 -> 74.112.38.136 TCP https > 52916 [SYN, ACK]
Seq=0 Ack=1 Win=14480 Len=0 MSS=1460 TSV=21847417 TSER=1260124448 WS=7
1832.688956 74.112.38.136 -> 10.252.168.252 TCP 52916 > https [ACK] Seq=1
Ack=1 Win=131760 Len=0 TSV=1260124633 TSER=21847417
1832.689008 74.112.38.136 -> 10.252.168.252 SSL Client Hello
1832.689027 10.252.168.252 -> 74.112.38.136 TCP https > 52916 [ACK] Seq=1
Ack=248 Win=15616 Len=0 TSV=21847472 TSER=1260124633
1832.689418 74.112.38.136 -> 10.244.46.81 TCP 49431 > https [SYN] Seq=0
Win=14600 Len=0 MSS=1460 TSV=21847472 TSER=0 WS=7
1833.686741 74.112.38.136 -> 10.244.46.81 TCP 49431 > https [SYN] Seq=0
Win=14600 Len=0 MSS=1460 TSV=21847722 TSER=0 WS=7
1835.690732 74.112.38.136 -> 10.244.46.81 TCP 49431 > https [SYN] Seq=0
Win=14600 Len=0 MSS=1460 TSV=21848223 TSER=0 WS=7
1839.694742 74.112.38.136 -> 10.244.46.81 TCP 49431 > https [SYN] Seq=0
Win=14600 Len=0 MSS=1460 TSV=21849224 TSER=0 WS=7
<On the web server node, 10.244.4681>
I really see no packets at all
So I double checked my configuration on the HAProxy node. I ran the
following iptables commands:
iptables -t mangle -N DIVERT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 111
iptables -t mangle -A DIVERT -j ACCEPT
ip rule add fwmark 111 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
And made sure ip forwarding was on:
echo 1 > /proc/sys/net/ipv4/conf/all/forwarding
echo 1 > /proc/sys/net/ipv4/conf/all/send_redirects
echo 1 > /proc/sys/net/ipv4/conf/eth0/send_redirects
And the firewall rules are:
$ sudo service iptables status
Table: mangle
Chain PREROUTING (policy ACCEPT)
num target prot opt source destination
1 DIVERT tcp -- 0.0.0.0/0 0.0.0.0/0 socket
Chain INPUT (policy ACCEPT)
num target prot opt source destination
Chain FORWARD (policy ACCEPT)
num target prot opt source destination
Chain OUTPUT (policy ACCEPT)
num target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
num target prot opt source destination
Chain DIVERT (1 references)
num target prot opt source destination
1 MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK set
0x6f
2 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
On the web ser
ver node, I made sure that my haproxy node was the default router
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt
Iface
0.0.0.0 10.252.168.252 0.0.0.0 UG 0 0 0
eth0
10.244.46.64 0.0.0.0 255.255.255.192 U 0 0 0
eth0
169.254.169.254 0.0.0.0 255.255.255.255 UH 0 0 0
eth0
and
ip route get 74.112.38.165
74.112.38.165 via 10.252.168.252 dev eth0 src 10.244.46.81
cache ipid 0xb9ee
And finally, no firewall rules on my webserver:
$ sudo service iptables status
Table: filter
Chain INPUT (policy ACCEPT)
num target prot opt source destination
Chain FORWARD (policy ACCEPT)
num target prot opt source destination
Chain OUTPUT (policy ACCEPT)
num target prot opt source destination
I am now at an impasse and don't know what else to check.
Does anyone have any pointers?
Ed
------------------------------
✉ Eduard Martinescu <[email protected]> | ✆ (585) 708-9685 | [image:
http://www.salsalabs.com] <http://www.salsalabs.com/> - ignite action. fuel
change.
