I am setting up a new system as a firewall using OpenBSD 6.8 current
-uname -a
OpenBSD fw1.lfz.net 6.8 GENERIC.MP#175 amd64.
I have 3 vlans 70,77,79 on the firewall using two em devices, em0 and
em1, in an aggregation to serve these vlans.
There is a Unifi switch which has 2 ports (where em0,em1 are attached)
set up to pass tagged vlans 70,77,79. The switch ip is 10.10.70.3.
I have a linux host setup on vans 70,77,79 and at address 77 -
10.10.70.77, 10.10.77.77,10.10.79.77.
So far i cannot communicate over the vlans. Before I vlanned these
subnets : ie only vlan 1 everywhere - communication worked fine.
So i do not believe there is a physical issue. The issues arose with the
introduction of the vlans. Is there a configuration issue that anyone
can spot?
Thank you for any help you can give.
Evidence:
ping on the firewall works locally
for n in 0 7 9 ; do ping -c 2 10.10.7${n}.1 ; done
PING 10.10.70.1 (10.10.70.1): 56 data bytes
64 bytes from 10.10.70.1: icmp_seq=0 ttl=255 time=0.037 ms
64 bytes from 10.10.70.1: icmp_seq=1 ttl=255 time=0.025 ms
--- 10.10.70.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.025/0.031/0.037/0.006 ms
PING 10.10.77.1 (10.10.77.1): 56 data bytes
64 bytes from 10.10.77.1: icmp_seq=0 ttl=255 time=0.038 ms
64 bytes from 10.10.77.1: icmp_seq=1 ttl=255 time=0.025 ms
--- 10.10.77.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.025/0.031/0.038/0.006 ms
PING 10.10.79.1 (10.10.79.1): 56 data bytes
64 bytes from 10.10.79.1: icmp_seq=0 ttl=255 time=0.038 ms
64 bytes from 10.10.79.1: icmp_seq=1 ttl=255 time=0.025 ms
--- 10.10.79.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.025/0.032/0.038/0.007 ms
ping to the switch does not work
ping -c 2 10.10.70.3
PING 10.10.70.3 (10.10.70.3): 56 data bytes
--- 10.10.70.3 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
ping to the linux host does not work.
ping -c 2 10.10.70.3
PING 10.10.70.3 (10.10.70.3): 56 data bytes
--- 10.10.70.3 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
[13:47:04] leonardz@fw1 etc>>for n in 0 7 9 ; do ping -c 2
10.10.7${n}.77 ; done
PING 10.10.70.77 (10.10.70.77): 56 data bytes
--- 10.10.70.77 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
PING 10.10.77.77 (10.10.77.77): 56 data bytes
--- 10.10.77.77 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
PING 10.10.79.77 (10.10.79.77): 56 data bytes
--- 10.10.79.77 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
I did the tests both with pfctl -e (enabled) and pfctl -d (disabled). It
made no difference
The setup is described below
Here is the setup:
===== hostname.aggr0
debug
trunkport em0
trunkport em1
up
inet 10.10.70.1/24
alias 10.10.77.1/24
alias 10.10.79.1/24
===== hostname.em0
up
===== hostname.em1
up
===== hostname.vlan70
parent aggr0 vnetid 70
10.10.70.0/24
===== hostname.vlan77
parent aggr0 vnetid 77
10.10.77.0/24
===== hostname.vlan79
parent aggr0 vnetid 79
10.10.79.0/24
Ifconfig -A shows the vlans are setup
===== aggr0
aggr0: flags=8847<UP,BROADCAST,DEBUG,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr fe:e1:ba:d0:f4:8c
index 6 priority 0 llprio 7
trunk: trunkproto lacp
trunk id: [(8000,fe:e1:ba:d0:f4:8c,0006,0000,0000),
(8000,e0:63:da:8e:78:d7,03E8,0000,0000)]
em0 lacp actor system pri 0x8000 mac fe:e1:ba:d0:f4:8c, key
0x6, port pri 0x8000 number 0x1
em0 lacp actor state
activity,aggregation,sync,collecting,distributing
em0 lacp partner system pri 0x8000 mac e0:63:da:8e:78:d7, key
0x3e8, port pri 0x1 number 0x9
em0 lacp partner state
activity,aggregation,sync,collecting,distributing
em0 port active,collecting,distributing
em1 lacp actor system pri 0x8000 mac fe:e1:ba:d0:f4:8c, key
0x6, port pri 0x8000 number 0x2
em1 lacp actor state
activity,aggregation,sync,collecting,distributing
em1 lacp partner system pri 0x8000 mac e0:63:da:8e:78:d7, key
0x3e8, port pri 0x1 number 0xa
em1 lacp partner state
activity,aggregation,sync,collecting,distributing
em1 port active,collecting,distributing
groups: aggr
media: Ethernet autoselect
status: active
inet 10.10.70.1 netmask 0xffffff00 broadcast 10.10.70.255
inet 10.10.77.1 netmask 0xffffff00 broadcast 10.10.77.255
inet 10.10.79.1 netmask 0xffffff00 broadcast 10.10.79.255
===== em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr fe:e1:ba:d0:f4:8c
index 1 priority 0 llprio 3
trunk: trunkdev aggr0
media: Ethernet autoselect (1000baseT full-duplex)
status: active
===== em1
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr fe:e1:ba:d0:f4:8c
index 2 priority 0 llprio 3
trunk: trunkdev aggr0
media: Ethernet autoselect (1000baseT full-duplex)
status: active
pfctl -sr
block return all
pass all flags S/SA
block return in on ! lo0 proto tcp from any to any port 6000:6010
block return out log proto tcp all user = 55
block return out log proto udp all user = 55
pass out log on aggr0 inet proto icmp from 10.10.70.0/24 to any label
"pings"
pass out log on aggr0 inet proto icmp from 10.10.77.0/24 to any label
"pings"
pass out log on aggr0 inet proto icmp from 10.10.79.0/24 to any label
"pings"
pass in on vlan70 all flags S/SA label "vlan70" tag vlan70
pass out on vlan70 all flags S/SA label "vlan70o" tag vlan70o
===== vlan70
vlan70: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr fe:e1:ba:d0:f4:8c
index 7 priority 0 llprio 3
encap: vnetid 70 parent aggr0 txprio packet rxprio outer
groups: vlan
media: Ethernet autoselect
status: active
inet 10.10.70.0 netmask 0xffffff00 broadcast 10.10.70.255
===== vlan77
vlan77: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr fe:e1:ba:d0:f4:8c
index 8 priority 0 llprio 3
encap: vnetid 77 parent aggr0 txprio packet rxprio outer
groups: vlan
media: Ethernet autoselect
status: active
inet 10.10.77.0 netmask 0xffffff00 broadcast 10.10.77.255
===== vlan79
vlan79: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr fe:e1:ba:d0:f4:8c
index 9 priority 0 llprio 3
encap: vnetid 79 parent aggr0 txprio packet rxprio outer
groups: vlan
media: Ethernet autoselect
status: active
inet 10.10.79.0 netmask 0xffffff00 broadcast 10.10.79.255
Routes
netstat -f inet -rn
Routing tables
Internet:
Destination Gateway Flags Refs Use Mtu Prio Iface
default 192.168.7.1 UGS 5 4045 - 8 re0
224/4 127.0.0.1 URS 0 116 32768 8 lo0
10.10.70/24 10.10.70.1 UCPn 1 7387 - 4 aggr0
10.10.70/24 10.10.70.0 UCPn 0 0 - 4 vlan70
10.10.70.0 fe:e1:ba:d0:f4:8c UHLl 0 0 - 1 vlan70
10.10.70.1 fe:e1:ba:d0:f4:8c UHLl 0 26 - 1 aggr0
10.10.70.3 e0:63:da:8e:78:d7 UHLc 0 7158 - 3 aggr0
10.10.70.255 10.10.70.1 UHPb 0 0 - 1 aggr0
10.10.70.255 10.10.70.0 UHPb 0 0 - 1 vlan70
10.10.77/24 10.10.77.1 UCPn 0 1 - 4 aggr0
10.10.77/24 10.10.77.0 UCPn 0 0 - 4 vlan77
10.10.77.0 fe:e1:ba:d0:f4:8c UHLl 0 0 - 1 vlan77
10.10.77.1 fe:e1:ba:d0:f4:8c UHLl 0 31 - 1 aggr0
10.10.77.255 10.10.77.1 UHPb 0 0 - 1 aggr0
10.10.77.255 10.10.77.0 UHPb 0 0 - 1 vlan77
10.10.79/24 10.10.79.1 UCPn 0 1 - 4 aggr0
10.10.79/24 10.10.79.0 UCPn 0 0 - 4 vlan79
10.10.79.0 fe:e1:ba:d0:f4:8c UHLl 0 0 - 1 vlan79
10.10.79.1 fe:e1:ba:d0:f4:8c UHLl 0 36 - 1 aggr0
10.10.79.255 10.10.79.1 UHPb 0 0 - 1 aggr0
10.10.79.255 10.10.79.0 UHPb 0 0 - 1 vlan79
127/8 127.0.0.1 UGRS 0 0 32768 8 lo0
127.0.0.1 127.0.0.1 UHhl 1 17 32768 1 lo0
192.168.7/24 192.168.7.4 UCn 1 0 - 4 re0
192.168.7.1 00:1b:21:18:88:72 UHLch 5 14796 - 3 re0
192.168.7.4 8c:ec:4b:7a:04:dc UHLl 0 184 - 1 re0
192.168.7.255 192.168.7.4 UHb 0 0 - 1 re0
the pf rules when pf enabled
pfctl -sr
block return all
pass all flags S/SA
block return in on ! lo0 proto tcp from any to any port 6000:6010
block return out log proto tcp all user = 55
block return out log proto udp all user = 55
pass out log on aggr0 inet proto icmp from 10.10.70.0/24 to any label
"pings"
pass out log on aggr0 inet proto icmp from 10.10.77.0/24 to any label
"pings"
pass out log on aggr0 inet proto icmp from 10.10.79.0/24 to any label
"pings"
pass in on vlan70 all flags S/SA label "vlan70" tag vlan70
pass out on vlan70 all flags S/SA label "vlan70o" tag vlan70o
sysctl for ip forwarding is set
net.inet.ip.forwarding=1
