On 20/10/15 21:47, Norman Henderson wrote: > Summary: packets received on one vlan interface, are replied to on > another vlan. This is specific to the network for which an address is > defined on vlan1; another network for which an address is defined on > vlan1:1 is working correctly. > > Environment: Ubuntu 14.04.3 LTS, Linux Kernel 3.13.0-65, Shorewall > version 4.5.21.6 > > A firewall has 2 physical interfaces eth0 and eth1. eth1 does not have > its own address because it hosts 3 vlan interfaces vlan1, vlan2, > vlan3. eth0 is a direct Internet connection with several addresses in > the same subnet. vlan1 has a main address 10.0.69.1/24 and a second > address 10.0.70.1/24 which we use for printers, switch management etc. > vlan3 10.1.10.248/24 is routed to another site which has alternative > internet connectivity. vlan2 192.168.11.248/24 is also routed to > another site but is currently hardly used. NB: None of this has been > changed anytime recently. > > The firewall was running fine until Thursday night when I removed a > vtun client definition which had been connecting over vlan3 and > changed routing so that the network at the other end, is connected > directly over the routed network rather than via vtun. There is still > a vtun server which receives connections from a server elsewhere in > the world. Unfortunately I don't have detailed notes of all changes > but I don't think I did anything unusual - mainly vtun and shorewall > config changes. > > MAIN SYMPTOM: Another box 10.0.69.20 on the same VLAN vlan1 sends a > Ping to the firewall as 10.0.69.1. The ping reply is generated > however, it is sent back to 10.0.69.20 on vlan3 instead of vlan1 and, > of course, does not arrive. The same symptom occurs for ssh, and dns, > and presumably all traffic incoming to address 10.0.69.1 on tcp/udp > ports FROM ALL OTHER DEVICES on vlan1 with an IP of 10.0.69.x/24. > > Ping, ssh, and dns incoming to ANY permitted address including > 10.0.69.1 on the firewall work from other origins: devices on vlan1 > with addresses 10.0.70.x, devices anywhere in our networks with > arbitrary addresses (as long as the firewall has a valid return > route). For example, 10.1.10.200, 10.1.0.211 and 10.1.15.254 are able > to access the firewall equally well addressing it as 10.1.10.248, > 10.0.69.1, or 10.0.70.1. > > The firewall itself can ping out to anywhere, in fact all outbound > functions work.
I won't confuse anyone with details (in case they are off-topic), but I thought it would be helpful to let you know I've been doing problem determination on what appears to be a similar issue, but with a different configuration. Naturally, I get shorewall log events for traffic between subnets that should NOT be allowed, but nothing is logged for those connections that are allowed. I believe it is not a shorewall problem, but something is going wrong quite low in the stack. The details seem to be frustratingly variable, but I often see redirect log messages to/from the host sending pings. I have many wireshark traces from a mirror port on my switch, but haven't yet spotted the root cause. In my research I found a reference to Linux being built on a "weak end system model" as defined in RFC1122, which apparently "leads to arp problems with multi-homed hosts". I haven't fully understood the theoretical issues yet, so I apologise if my comments are not relevant to your situation. However, in case it is relevant I thought it best to mention quickly. This mailing list always gives good advice, but my problem (and perhaps yours too) doesn't seem to be attributable to shorewall, or fixable with it. HTH, Brian >>From 10.0.69.20 I can arping 10.0.69.1 and I get a valid reply, the > MAC of vlan1 i.e. of eth1. As expected, 10.0.69.20 cannot arping > 10.0.70.1 (I guess that's due to arp_ignore=2). > > The firewall IS routing packets coming from arbitrary 10.0.69.x/24 > addresses (including 10.0.69.20) that are destined for the outside > world via vlan3 or eth0 and vice-versa. For example, 10.0.69.20 can > ping from 10.0.69.20 to 93.184.220.29 and gets a reply; both the ping > and reply are visible on the firewall on vlan1 and also on the > upstream interface vlan3. > > Devices on vlan1with an IP of 10.0.70.x/24 (if you prefer, vlan1:1 > addresses) can ping 10.0.70.1. In fact, devices on vlan1 with an IP of > 10.0.69.x can ping 10.0.70.1 (those packets are being being routed via > the firewall and are visible with tcpdump listening on vlan1). > > I have checked the switching topology. It consists of a gigabit > backbone, for which all of the ports that interconnect switches are > set for PVID 1, and are Tagged members of vlan 1, 2, and 3. Almost all > other ports in the network are set to Untagged on vlan 1 only, with > pvid 1. One port only on one switch is Untagged vlan2 (only), pvid2 > and a couple of ports on two switches are Untagged vlan3 (only), > pvid3. > > Shorewall zones: > Fwall firewall > CEM09 ipv4 > CEM01 ipv4 > CEM11 ipv4 > CEM50 ipv4 > NCI01 ipv4 > MAF01 ipv4 > UNI01 ipv4 > > Shorewall interfaces: > ?FORMAT 2 > #ZONE INTERFACE OPTIONS > CEM09 tun0 routeback,optional > CEM50 vlan3 routeback > UNI01 ppp0 optional > - vlan1 dhcp,arp_ignore=2 > NCI01 eth0 routefilter=1 > MAF01 vlan2 routeback #was wlan0 > > Shorewall hosts: > #ZONE HOSTS OPTIONS > CEM01 vlan1:10.0.69.0/24 routeback > CEM11 vlan1:10.0.70.0/24 routeback > > For test purposes I added three lines to the front of > shorewall/policy, here is the complete file: > Fwall all ACCEPT > CEM01 $FW ACCEPT > CEM11 $FW ACCEPT > > CEM01 CEM11 ACCEPT > CEM11 CEM01 ACCEPT > CEM11 CEM09 ACCEPT > CEM01 all REJECT info > CEM11 CEM50 REJECT info > CEM11 MAF01 REJECT info > CEM11 NCI01 REJECT > CEM11 UNI01 REJECT > CEM50 all REJECT info > CEM09 all REJECT info > NCI01 all REJECT > MAF01 all REJECT info > UNI01 all REJECT > > The rules are a mess but, given the wide-open policy in the first > three lines above, I don't believe there is a filtering problem. > > TCPDUMPs showing pings coming in on vlan1 and going out on vlan3: > > # tcpdump -i vlan1 host 10.0.69.20 and icmp > listening on vlan1, link-type EN10MB (Ethernet), capture size 65535 bytes > 08:37:24.521767 IP 10.0.69.20 > cem01fw-clean.ceml.net: ICMP echo > request, id 18086, seq 1, length 64 > 08:37:25.521082 IP 10.0.69.20 > cem01fw-clean.ceml.net: ICMP echo > request, id 18086, seq 2, length 64 > 08:37:26.529489 IP 10.0.69.20 > cem01fw-clean.ceml.net: ICMP echo > request, id 18086, seq 3, length 64 > ^C // note, there are no replies on vlan1 // > # tcpdump -i vlan3 host 10.0.69.20 and icmp > listening on vlan3, link-type EN10MB (Ethernet), capture size 65535 bytes > 08:37:38.667660 IP cem01fw-clean.ceml.net > 10.0.69.20: ICMP echo > reply, id 18098, seq 1, length 64 > 08:37:38.675847 IP cem01fw-clean.ceml.net > 10.0.69.20: ICMP echo > reply, id 18098, seq 1, length 64 > 08:37:39.667679 IP cem01fw-clean.ceml.net > 10.0.69.20: ICMP echo > reply, id 18098, seq 2, length 64 > ^C // note, ONLY replies appear on vlan3 // > > IPTABLES TRACE showing the same thing: > Oct 17 08:30:14 cem01fw kernel: [65995.522023] TRACE: > raw:PREROUTING:policy:14 IN=vlan1 OUT= > MAC=00:9c:02:aa:37:dd:78:e3:b5:fc:3c:22:08:00 SRC=10.0.69.20 > DST=10.0.69.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=28169 DF PROTO=ICMP > TYPE=8 CODE=0 ID=17920 SEQ=9 > Oct 17 08:30:14 cem01fw kernel: [65995.522046] TRACE: > mangle:PREROUTING:policy:5 IN=vlan1 OUT= > MAC=00:9c:02:aa:37:dd:78:e3:b5:fc:3c:22:08:00 SRC=10.0.69.20 DST= > 10.0.69.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=28169 DF PROTO=ICMP > TYPE=8 CODE=0 ID=17920 SEQ=9 > Oct 17 08:30:14 cem01fw kernel: [65995.522058] TRACE: > mangle:INPUT:policy:1 IN=vlan1 OUT= > MAC=00:9c:02:aa:37:dd:78:e3:b5:fc:3c:22:08:00 SRC=10.0.69.20 DST=10.0. > 69.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=28169 DF PROTO=ICMP TYPE=8 > CODE=0 ID=17920 SEQ=9 > Oct 17 08:30:14 cem01fw kernel: [65995.522066] TRACE: > filter:INPUT:rule:2 IN=vlan1 OUT= > MAC=00:9c:02:aa:37:dd:78:e3:b5:fc:3c:22:08:00 SRC=10.0.69.20 > DST=10.0.69.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=28169 DF PROTO=ICMP > TYPE=8 CODE=0 ID=17920 SEQ=9 > Oct 17 08:30:14 cem01fw kernel: [65995.522085] TRACE: > filter:vlan1_in:rule:4 IN=vlan1 OUT= > MAC=00:9c:02:aa:37:dd:78:e3:b5:fc:3c:22:08:00 SRC=10.0.69.20 > DST=10.0.69.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=28169 DF PROTO=ICMP > TYPE=8 CODE=0 ID=17920 SEQ=9 > Oct 17 08:30:14 cem01fw kernel: [65995.522089] TRACE: > filter:CEM012Fwall:rule:1 IN=vlan1 OUT= > MAC=00:9c:02:aa:37:dd:78:e3:b5:fc:3c:22:08:00 SRC=10.0.69.20 > DST=10.0.69.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=28169 DF PROTO=ICMP > TYPE=8 CODE=0 ID=17920 SEQ=9 > Oct 17 08:30:14 cem01fw kernel: [65995.522106] TRACE: > raw:OUTPUT:policy:13 IN= OUT=vlan3 SRC=10.0.69.1 DST=10.0.69.20 LEN=84 > TOS=0x00 PREC=0x00 TTL=64 ID=50736 PROTO=ICMP TYPE=0 CODE=0 ID=1792 > 0 SEQ=9 > Oct 17 08:30:14 cem01fw kernel: [65995.522110] TRACE: > mangle:OUTPUT:policy:1 IN= OUT=vlan3 SRC=10.0.69.1 DST=10.0.69.20 > LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=50736 PROTO=ICMP TYPE=0 CODE=0 > ID=17920 SEQ=9 > Oct 17 08:30:14 cem01fw kernel: [65995.522113] TRACE: > filter:OUTPUT:policy:3 IN= OUT=vlan3 SRC=10.0.69.1 DST=10.0.69.20 > LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=50736 PROTO=ICMP TYPE=0 CODE=0 > ID=17920 SEQ=9 > Oct 17 08:30:14 cem01fw kernel: [65995.522115] TRACE: > mangle:POSTROUTING:policy:2 IN= OUT=vlan3 SRC=10.0.69.1 DST=10.0.69.20 > LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=50736 PROTO=ICMP TYPE=0 CODE=0 > ID=17920 SEQ=9 > > I'm so far out of my depth here, the water above me is black. Help!!! > > > > ------------------------------------------------------------------------------ > > > > _______________________________________________ > Shorewall-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/shorewall-users > ------------------------------------------------------------------------------ _______________________________________________ Shorewall-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/shorewall-users
