I had a similar speed drop on an Edge Router 4. I don’t know if it’s the same situation on the Lite, but I believe it’s expected due to hardware acceleration support (or lack of) and single core performance on the pf side.
Scott > On Oct 4, 2020, at 17:24, Amarendra Godbole <amarendra.godb...@gmail.com> > wrote: > > Sorry I forgot including "ifconfig" output: > > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 32768 > index 5 priority 0 llprio 3 > groups: lo > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 > inet 127.0.0.1 netmask 0xff000000 > > cnmac0: flags=808843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,AUTOCONF4> mtu > 1500 > lladdr a8:28:dc:cc:2e:6f > index 1 priority 0 llprio 3 > groups: egress > media: Ethernet autoselect (1000baseT full-duplex,master) > status: active > inet 73.xx.xx.xx netmask 0xfffffe00 broadcast 73.xx.xx.255 > > cnmac1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> > mtu 1500 > lladdr 78:8a:20:46:a8:c1 > index 2 priority 0 llprio 3 > media: Ethernet autoselect (1000baseT full-duplex) > status: active > > cnmac2: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> > mtu 1500 > lladdr 78:8a:20:46:a8:c2 > index 3 priority 0 llprio 3 > media: Ethernet autoselect (none) > status: no carrier > enc0: flags=0<> > index 4 priority 0 llprio 3 > groups: enc > status: active > > bridge0: flags=41<UP,RUNNING> > index 6 llprio 3 > groups: bridge > priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp > cnmac2 flags=7<LEARNING,DISCOVER,BLOCKNONIP> > port 3 ifpriority 0 ifcost 0 > cnmac1 flags=7<LEARNING,DISCOVER,BLOCKNONIP> > port 2 ifpriority 0 ifcost 0 > vether0 flags=7<LEARNING,DISCOVER,BLOCKNONIP> > port 7 ifpriority 0 ifcost 0 > > vether0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 > lladdr fe:e1:ba:d0:c8:a9 > index 7 priority 0 llprio 3 > groups: vether > media: Ethernet autoselect > status: active > inet 192.168.10.1 netmask 0xffffff00 broadcast 192.168.10.255 > > pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33136 > index 8 priority 0 llprio 3 > groups: pflog > >> On Sun, Oct 4, 2020 at 2:22 PM Amarendra Godbole >> <amarendra.godb...@gmail.com> wrote: >> >> Hi misc@ >> >> I recently introduced an OpenBSD firewall inline and noticed a >> reduction in overall download speeds. I am trying to understand why >> this may be so. The firewall is Ubiquiti ERL running 6.7 release. >> Internet connection is Comcast xfinity via cable modem, plan 200 >> Mbits/s down and 10 Mbits/s up. Details follow: >> >> 1. config #1: MacBook - Linksys WRT1200AC - xfinity cable modem >> (speed: ~210 Mbits/s down, 6 Mbits/s up) >> 2. config #2: MacBook - Linksys WRT1200AC - Ubiquiti ERL - xfinity >> cable modem (speed: ~90 MBits down, 6 Mbits/s up) >> 3. config #3 (Line speed): MacBook wired to cable modem (~230 Mbits/s >> down, ~8 Mbits/s up). >> >> Linksys is running latest OpenWrt, and speed tests were run on MacBook >> connected wired to Linksys. It was difficult to try tcpbench since the >> setup was cumbersome, and iperf3 public servers end up being busy more >> often than not (and threads on misc@ indicated iperf3 wasn't as >> reliable either). Test numbers come from speedtest.net and >> speed.cloudflare.com. While I realize this speed test is hardly >> accurate, I have tried to maintain the same configuration (no ERL and >> inline ERL) to obtain relative numbers. >> >> I am trying to understand the reduction from 210 Mbits/s down to 90 >> Mbits/s down between config #1 and config #2 above. The slowdown is >> not noticeable to family, so this is more of my intellectual curiosity >> than screams over a buffering video! :-) >> >> Relevant system information (dmesg, etc.) below. All sysctl values >> attached as sysctl.txt I gathered it by reading similar threads on >> misc@. If I missed anything, please let me know. Thanks in advance. >> >> -Amarendra >> >> >> dmesg: >> >> Copyright (c) 1982, 1986, 1989, 1991, 1993 >> The Regents of the University of California. All rights reserved. >> Copyright (c) 1995-2020 OpenBSD. All rights reserved. >> https://www.OpenBSD.org >> OpenBSD 6.7 (GENERIC.MP) #134: Thu May 7 16:05:06 MDT 2020 >> dera...@octeon.openbsd.org:/usr/src/sys/arch/octeon/compile/GENERIC.MP >> real mem = 536870912 (512MB) >> avail mem = 506740736 (483MB) >> mainbus0 at root: board 20002 rev 2.18 >> cpu0 at mainbus0: CN50xx CPU rev 0.1 500 MHz, Software FP emulation >> cpu0: cache L1-I 32KB 4 way D 16KB 64 way, L2 128KB 8 way >> cpu1 at mainbus0: CN50xx CPU rev 0.1 500 MHz, Software FP emulation >> cpu1: cache L1-I 32KB 4 way D 16KB 64 way, L2 128KB 8 way >> clock0 at mainbus0: int 5 >> octcrypto0 at mainbus0 >> iobus0 at mainbus0 >> simplebus0 at iobus0: "soc" >> octciu0 at simplebus0 >> octsmi0 at simplebus0 >> octpip0 at simplebus0 >> octgmx0 at octpip0 interface 0 >> cnmac0 at octgmx0: RGMII, address 78:8a:20:46:a8:c0 >> atphy0 at cnmac0 phy 7: AR8035 10/100/1000 PHY, rev. 2 >> cnmac1 at octgmx0: RGMII, address 78:8a:20:46:a8:c1 >> atphy1 at cnmac1 phy 6: AR8035 10/100/1000 PHY, rev. 2 >> cnmac2 at octgmx0: RGMII, address 78:8a:20:46:a8:c2 >> atphy2 at cnmac2 phy 5: AR8035 10/100/1000 PHY, rev. 2 >> com0 at simplebus0: ns16550a, 64 byte fifo >> com0: console >> dwctwo0 at iobus0 base 0x1180068000000 irq 56 >> usb0 at dwctwo0: USB revision 2.0 >> uhub0 at usb0 configuration 1 interface 0 "Octeon DWC2 root hub" rev >> 2.00/1.00 addr 1 >> octrng0 at iobus0 base 0x1400000000000 irq 0 >> /dev/ksyms: Symbol table not valid. >> umass0 at uhub0 port 1 configuration 1 interface 0 "Lexar USB Flash >> Drive" rev 2.10/11.00 addr 2 >> umass0: using SCSI over Bulk-Only >> scsibus0 at umass0: 2 targets, initiator 0 >> sd0 at scsibus0 targ 1 lun 0: <Lexar, USB Flash Drive, 1100> removable >> serial.21c40cd1719080003000 >> sd0: 30526MB, 512 bytes/sector, 62517248 sectors >> vscsi0 at root >> scsibus1 at vscsi0: 256 targets >> softraid0 at root >> scsibus2 at softraid0: 256 targets >> boot device: sd0 >> root on sd0a (2124441bc835a462.a) swap on sd0b dump on sd0b >> WARNING: No TOD clock, believing file system. >> WARNING: CHECK AND RESET THE DATE! >> >> >> pftcl -s: >> >> match in all scrub (no-df random-id max-mss 1440) >> block drop in quick on ! cnmac0 inet from xx.xx.xx.xx/23 to any >> block drop in quick inet from xx.xx.xx.xx to any >> block drop all >> pass out quick on egress inet from (vether0:network) to any flags S/SA >> nat-to (egress) round-robin >> pass out quick inet all flags S/SA >> pass in on vether0 inet all flags S/SA >> pass in on cnmac1 inet all flags S/SA >> pass in on cnmac2 inet all flags S/SA >> >> pfctl -si: >> >> Status: Enabled for 3 days 18:14:11 Debug: err >> >> Interface Stats for egress IPv4 IPv6 >> >> Bytes In 64537867779 0 >> Bytes Out 7005140381 0 >> Packets In >> Passed 56024370 0 >> Blocked 43050 0 >> Packets Out >> Passed 22271061 0 >> Blocked 0 0 >> >> State Table Total Rate >> current entries 847 >> half-open tcp 23 >> searches 158989755 489.4/s >> inserts 1095307 3.4/s >> removals 1094460 3.4/s >> Counters >> match 1343178 4.1/s >> bad-offset 0 0.0/s >> fragment 0 0.0/s >> short 1 0.0/s >> normalize 0 0.0/s >> memory 0 0.0/s >> bad-timestamp 0 0.0/s >> congestion 41 0.0/s >> ip-option 26164 0.1/s >> proto-cksum 0 0.0/s >> state-mismatch 10758 0.0/s >> state-insert 0 0.0/s >> state-limit 0 0.0/s >> src-limit 0 0.0/s >> synproxy 0 0.0/s >> translate 0 0.0/s >> no-route 0 0.0/s >> >> pfctl -s memory: >> >> states hard limit 100000 >> src-nodes hard limit 10000 >> frags hard limit 16384 >> tables hard limit 1000 >> table-entries hard limit 200000 >> pktdelay-pkts hard limit 10000 >> >> The netlivlocks value keeps on increasing regularly: >> kern.netlivelocks=57911 >> >> netstat -m: >> >> 1009 mbufs in use: >> 917 mbufs allocated to data >> 5 mbufs allocated to packet headers >> 87 mbufs allocated to socket names and addresses >> 801/7256 mbuf 2048 byte clusters in use (current/peak) >> 0/15 mbuf 2112 byte clusters in use (current/peak) >> 0/24 mbuf 4096 byte clusters in use (current/peak) >> 0/8 mbuf 8192 byte clusters in use (current/peak) >> 0/0 mbuf 9216 byte clusters in use (current/peak) >> 0/0 mbuf 12288 byte clusters in use (current/peak) >> 0/0 mbuf 16384 byte clusters in use (current/peak) >> 0/8 mbuf 65536 byte clusters in use (current/peak) >> 6512/17088/131072 Kbytes allocated to network (current/peak/max) >> 0 requests for memory denied >> 0 requests for memory delayed >> 0 calls to protocol drain routines >> >> netstat -i: >> >> Name Mtu Network Address Ipkts Ifail Opkts Ofail >> Colls >> lo0 32768 <Link> 198 0 198 0 >> 0 >> lo0 32768 localhost/1 localhost 198 0 198 0 >> 0 >> lo0 32768 fe80::%lo0/ fe80::1%lo0 198 0 198 0 >> 0 >> lo0 32768 127/8 localhost 198 0 198 0 >> 0 >> cnmac0 1600 <Link> a8:28:dc:cc:2e:6f 56088774 0 22283491 2688 >> 0 >> cnmac0 1600 73.231.60/2 c-73-231-60-128.h 56088774 0 22283491 2688 >> 0 >> cnmac1 1600 <Link> 78:8a:20:46:a8:c1 23646497 4 56569853 48 >> 0 >> cnmac2 1600 <Link> 78:8a:20:46:a8:c2 14823 0 226198 226198 >> 0 >> enc0* 0 <Link> 0 0 0 0 >> 0 >> bridge0 1500 <Link> 23187238 0 57022219 0 >> 0 >> vether0 32768 <Link> fe:e1:ba:d0:c8:a9 23056709 0 56795991 0 >> 0 >> vether0 32768 192.168.10/ 192.168.10.1 23056709 0 56795991 0 >> 0 >> pflog0 33136 <Link> 0 0 26171 0 >> 0 >
