On 17/03/13 15:09, Tom Eastep wrote: I assume your focus on what iss happening is an indication that I do have the correct basic configuration?
>> >> $ wget -O - http://gb.archive.ubuntu.com/ubuntu/ >> >> tries to connect to each of the round-robin IPs associated with the >> hostname but eventually fails even though the tcpdump and iptrace >> logging shows responses returning via tun0 to Jeeves. > > Your system log is likely full of 'Martian' messages. Please confirm. Messages containing "martian source" or "ll header" are being directed to /var/log/network.log. Checking that file shows it was, some of the times I was experimenting with this configuration, seeing lots of "martian source" warnings, although not within the couple of hours. >> >> This affects LAN clients *and* direct connections from Jeeves whether >> or not the Squid transparent proxy is active, which leads me to >> believe I'm missing something in the Shorewall configuration. >> > > Probably need to set both log_martians and routefilter to 0 for tun0 in > /etc/shorewall/interfaces. In the data-centre, the host "pella": $ for if in eth0 eth0:0 tun0; do ifconfig $if | head -n 2; done eth0 Link encap:Ethernet HWaddr f2:3c:91:96:85:58 inet addr:109.74.197.121 Bcast:109.74.197.255 Mask:255.255.255.0 eth0:0 Link encap:Ethernet HWaddr f2:3c:91:96:85:58 inet addr:109.74.197.122 Bcast:109.74.197.255 Mask:255.255.255.0 tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:10.254.252.1 P-t-P:10.254.252.2 Mask:255.255.255.255 Back on host "Jeeves": $ for if in ppp0 tun0 br0; do ifconfig $if | head -n 2; done ppp0 Link encap:Point-to-Point Protocol inet addr:82.71.24.87 P-t-P:62.3.83.21 Mask:255.255.255.255 tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:10.254.252.18 P-t-P:10.254.252.17 Mask:255.255.255.255 br0 Link encap:Ethernet HWaddr 00:04:23:bd:f6:2c inet addr:10.254.251.1 Bcast:10.254.251.255 Mask:255.255.255.0 /etc/shorewall/interfaces: #ZONE INTERFACE BROADCAST OPTIONS net ppp0 - routefilter,tcpflags,logmartians,nosmurfs,mss=1452 net ppp1 - routefilter,tcpflags,logmartians,nosmurfs,mss=1452 loc dsl0 detect routefilter,tcpflags,logmartians,nosmurfs loc br0 detect dhcp,routefilter,tcpflags,logmartians,nosmurfs loc wlan0 detect dhcp,routefilter,tcpflags,logmartians,nosmurfs loc eth9 detect routefilter,tcpflags,logmartians,nosmurfs dc tun0 detect routefilter=0,tcpflags,logmartians=0,nosmurfs /etc/shorewall/masq: #INTERFACE SOURCE ADDRESS PROTO PORT(S) IPSEC MARK ppp0 10.0.0.0/8,169.254.0.0/16,172.16.0.0/12,192.168.0.0/16 82.71.24.87 #ppp1 10.0.0.0/8,169.254.0.0/16,172.16.0.0/12,192.168.0.0/16 82.71.24.87 eth9 br0 #tun0 10.0.0.0/8,127.0.0.0/8 10.254.252.18 #tun0 br0 Note: dsl0 and ppp1 are hold-overs from the former ADSL line before a recent upgrade to VDSL. dsl0 is an ethernet interface on an internal PCI ADSL2+ modem. ppp1 is not used but its configuration is maintained in the system just-in-case. $ ip rule ls 0: from all lookup local 10000: from all fwmark 0x50/0xff lookup VPN 32766: from all lookup main 32767: from all lookup default $ ip route ls table VPN default via 10.254.252.17 dev tun0 src 10.254.252.18 10.254.1.0 via 10.254.251.50 dev br0 10.254.251.0/24 dev br0 proto kernel scope link src 10.254.251.1 10.254.252.1 via 10.254.252.17 dev tun0 10.254.252.17 dev tun0 scope link src 10.254.252.18 109.74.192.20 via 10.254.252.17 dev tun0 109.74.193.20 via 10.254.252.17 dev tun0 $ ip route ls default via 62.3.83.21 dev ppp0 10.254.1.0 via 10.254.251.50 dev br0 10.254.248.0/30 dev eth9 proto kernel scope link src 10.254.248.2 10.254.251.0/24 dev br0 proto kernel scope link src 10.254.251.1 10.254.252.1 via 10.254.252.17 dev tun0 10.254.252.17 dev tun0 scope link src 10.254.252.18 62.3.83.21 dev ppp0 proto kernel scope link src 82.71.24.87 109.74.192.20 via 10.254.252.17 dev tun0 109.74.193.20 via 10.254.252.17 dev tun0 Note: the last two entries are the DNS server routes added by openvpn. Testing fails with: $ wget -O - http://gb.archive.ubuntu.com/ubuntu/ --2013-03-17 16:25:11-- http://gb.archive.ubuntu.com/ubuntu/ Resolving gb.archive.ubuntu.com (gb.archive.ubuntu.com)... 91.189.92.201, 91.189.92.202, 194.169.254.10, ... Connecting to gb.archive.ubuntu.com (gb.archive.ubuntu.com)|91.189.92.201|:80... ^C $ sudo tcpdump -ni tun0 port 80 or tcp port 3128 listening on tun0, link-type RAW (Raw IP), capture size 65535 bytes 16:25:12.024613 IP 82.71.24.87.39176 > 91.189.92.201.80: Flags [S], seq 1971087366, win 14520, options [mss 1452,sackOK,TS val 168165226 ecr 0,nop,wscale 4], length 0 16:25:13.023137 IP 82.71.24.87.39176 > 91.189.92.201.80: Flags [S], seq 1971087366, win 14520, options [mss 1452,sackOK,TS val 168165476 ecr 0,nop,wscale 4], length 0 Notice the source IP on tun0 is the IP of ppp0. This was why I added the MASQ entry: /etc/shorewall/masq #INTERFACE SOURCE ADDRESS PROTO PORT(S) IPSEC MARK ppp0 10.0.0.0/8,169.254.0.0/16,172.16.0.0/12,192.168.0.0/16 82.71.24.87 #ppp1 10.0.0.0/8,169.254.0.0/16,172.16.0.0/12,192.168.0.0/16 82.71.24.87 eth9 br0 tun0 10.0.0.0/8,127.0.0.0/8 10.254.252.18 #tun0 br0 $ sudo iptables -t nat -nL | awk '$0~/(POSTROUTING|(ppp0|tun0)_masq)/ { show=1} $0~/^$/ {show=0} {if (show) print $0}' Chain POSTROUTING (policy ACCEPT) target prot opt source destination ppp0_masq all -- 0.0.0.0/0 0.0.0.0/0 eth9_masq all -- 0.0.0.0/0 0.0.0.0/0 tun0_masq all -- 0.0.0.0/0 0.0.0.0/0 Chain ppp0_masq (1 references) target prot opt source destination SNAT all -- 10.0.0.0/8 0.0.0.0/0 to:82.71.24.87 SNAT all -- 169.254.0.0/16 0.0.0.0/0 to:82.71.24.87 SNAT all -- 172.16.0.0/12 0.0.0.0/0 to:82.71.24.87 SNAT all -- 192.168.0.0/16 0.0.0.0/0 to:82.71.24.87 Chain tun0_masq (1 references) target prot opt source destination SNAT all -- 10.0.0.0/8 0.0.0.0/0 to:10.254.252.18 SNAT all -- 127.0.0.0/8 0.0.0.0/0 to:10.254.252.18 But the same result: $ sudo tcpdump -ni tun0 port 80 or tcp port 3128 listening on tun0, link-type RAW (Raw IP), capture size 65535 bytes 16:25:12.024613 IP 82.71.24.87.39176 > 91.189.92.201.80: Flags [S], seq 1971087366, win 14520, options [mss 1452,sackOK,TS val 168165226 ecr 0,nop,wscale 4], length 0 The 'iptrace' seems to show that SNAT isn't happening: $ clear && tail -n 0 -f /var/log/kern.log | tee /tmp/nf_Trace.log | egrep '(91\.189\.92).*(UID=13|(SPT|DPT)=80|0x50)' Mar 17 16:36:45 jeeves kernel: [673654.416999] TRACE: raw:OUTPUT:policy:3 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 Mar 17 16:36:45 jeeves kernel: [673654.417032] TRACE: mangle:OUTPUT:rule:2 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 Mar 17 16:36:45 jeeves kernel: [673654.417054] TRACE: mangle:tcout:rule:1 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 Mar 17 16:36:45 jeeves kernel: [673654.417075] TRACE: mangle:tcout:return:2 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:45 jeeves kernel: [673654.417090] TRACE: mangle:OUTPUT:policy:3 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:45 jeeves kernel: [673654.417126] TRACE: nat:OUTPUT:policy:1 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:45 jeeves kernel: [673654.417150] TRACE: filter:OUTPUT:rule:1 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:45 jeeves kernel: [673654.417191] TRACE: filter:fw2net:rule:6 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:45 jeeves kernel: [673654.417204] TRACE: mangle:POSTROUTING:rule:1 IN= OUT=tun0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:45 jeeves kernel: [673654.417224] TRACE: mangle:tcpost:return:1 IN= OUT=tun0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:45 jeeves kernel: [673654.417242] TRACE: mangle:POSTROUTING:policy:2 IN= OUT=tun0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:45 jeeves kernel: [673654.417254] TRACE: nat:POSTROUTING:rule:3 IN= OUT=tun0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:45 jeeves kernel: [673654.417269] TRACE: nat:tun0_masq:return:3 IN= OUT=tun0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:45 jeeves kernel: [673654.417281] TRACE: nat:POSTROUTING:policy:4 IN= OUT=tun0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33201 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A4AC0000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:46 jeeves kernel: [673655.416051] TRACE: raw:OUTPUT:policy:3 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33202 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A5A60000000001030304) UID=1000 GID=1000 Mar 17 16:36:46 jeeves kernel: [673655.416074] TRACE: mangle:OUTPUT:rule:2 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33202 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A5A60000000001030304) UID=1000 GID=1000 Mar 17 16:36:46 jeeves kernel: [673655.416088] TRACE: mangle:tcout:rule:1 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33202 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A5A60000000001030304) UID=1000 GID=1000 Mar 17 16:36:46 jeeves kernel: [673655.416100] TRACE: mangle:tcout:return:2 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33202 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A5A60000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:46 jeeves kernel: [673655.416112] TRACE: mangle:OUTPUT:policy:3 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33202 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A5A60000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:46 jeeves kernel: [673655.416131] TRACE: filter:OUTPUT:rule:1 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33202 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A5A60000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:46 jeeves kernel: [673655.416165] TRACE: filter:fw2net:rule:6 IN= OUT=ppp0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33202 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A5A60000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:46 jeeves kernel: [673655.416177] TRACE: mangle:POSTROUTING:rule:1 IN= OUT=tun0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33202 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A5A60000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:46 jeeves kernel: [673655.416190] TRACE: mangle:tcpost:return:1 IN= OUT=tun0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33202 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A5A60000000001030304) UID=1000 GID=1000 MARK=0x50 Mar 17 16:36:46 jeeves kernel: [673655.416202] TRACE: mangle:POSTROUTING:policy:2 IN= OUT=tun0 SRC=82.71.24.87 DST=91.189.92.201 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33202 DF PROTO=TCP SPT=39245 DPT=80 SEQ=974071069 ACK=0 WINDOW=14520 RES=0x00 SYN URGP=0 OPT (020405AC0402080A0A08A5A60000000001030304) UID=1000 GID=1000 MARK=0x50 ^C Note: UID=13 is the 'proxy' user that squid3 runs as. At one point late yesterday I had a configuration that was returning replies via tun0 although the processes on Jeeves were not seeing them, but I've not been able to rediscover that config today. That had the correct source IP (10.254.252.18). Tiredness and confusion don't help! ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar _______________________________________________ Shorewall-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/shorewall-users
