On 11.11.2019. 13:42, Szél Gábor wrote:
Hello @misc,
Hi,
We have an interesting problem, we run a lot of OpenBSD router/firewalls
in many places.
We have a larger network than our client, 300-400 local wired or
wireless endpoint, 20+ VLAN, 20+ switches.
Network structure:
* Main switch - 2x Cisco Nexus 3k switch in HA mode (vPC dedicated
2x40Gbit Peer link, keepalive link)
* access switch - 10+ Cisco 3750X + C3KX-SM-10G 10Gbit module.
some 3750x stacked (2 or 3 switch)
* Main and access switches have redundant 10Gbit fiber link (LACP)
* when is possible jumbo frame is enabled (mtu 9000)
Firewall/router:
* 2x Dell 2950 - 2x Xeon X5460 (8 core), 8Gb Memory, 2x10Gbit SFP+
network card
hardware is really old, if you can, buy something newer
* redundant design - CARP, pfsync, ifstated, etc .... master-backup>
configuration
* HP NC550SFP network card, oce driver (mtu 9000)
if you can change oce with ix. ixl is not so bad .. .
* dual SFP+ port have LACP link to Nexus switches (2x10Gbit access
link) - use openbsd trunk interface
* all vlan used openbsd pseudo-device over trunk interface (VLANs not
have have IP address, only up)
update to openbsd 6.6 or snapshot and insted of trunk use aggr. why vlan
interfaces don't have ip address ?
* all network subnets defined in CARP interfaces, only managment VLAN
have address on VLAN interface.
who is parent interface for carp ?
* some vether virtual interface for VPN, DNS, etc ...
vether implies that you have bridge? bridge is slow..
* some tun and tap interface for VPN
* enc interface for ipsec
ipsec is performance killer big time ... even for traffic that doesn't
go through ipsec tunnel .. if you can move ipsec or any vpn stuff to
other boxes that you speed up your firewalls ...
* one bridge interface for openVPN (during termination)
vether is in that bridge?
* OpenBSD 6.3 64bit
please, update boxes regularly.. you have carp and pfsync, you can do
that without any problem ..
PF:
* global block rule (block all)
* ruleset-optimization none
* optimization aggressive
* reassemble no
* block-policy drop
* scrub enabled
* antispoof enabled
* regulating traffic between subnets with pf pass in/out rules
* pf.conf currently 1500+ lines
* the number of connections during the day in PF 10 000+
Problem:
We see that network traffic is limited to 1Gbit on firewall. Not in one
link, not IP-to-IP, to the whole firewall!
yes ... ipsec, trunk, pf are for whole firewall .. and even if you have
fastest box in the world you will not get performance that you want ..
example:
* i make test traffic form VLAN 2 to VLAN 12 witch iperf.
test PC-s have 1Gbit ethernet cards.
Speed is okay, ~800Mbit/sec
* i make anoter traffic from VLAN 2 to VLan20 with iperf, from another
PC-s
(they also have 1gbit ethernet cards)
speed is not good! ~60-80Mbit/sec
* if i stopped first speed test (2->12), second test speed is okay!
(2->20)
* but i make test from completely different VLANs, 2->12 and 20->30,
the result is so.
if you disable pf on vlan intefaces (set skip on vlan2/vlan12) do you
get better performace?
and after that for disable ipsec and try testing again... do you see
differences ?
This is firewall (openbsd) limitation, but we don't understand why?
I know openbsd VLAN interface has a speed problem, this is it?
not in OpenBSD 6.6
I know it's so difficult to make a mistake from some information, what
should we look at?
OpenBSD is great router and firewall that can do so much for you .. but
please you really need to rethink your hardware and setup ..
