On Tue, Oct 25, 2016 at 12:50:50AM +0200, Mike Belopuhov wrote: > On Tue, Oct 25, 2016 at 00:22 +0200, Hrvoje Popovski wrote: > > On 24.10.2016. 23:36, Mike Belopuhov wrote: > > > On Mon, Oct 24, 2016 at 19:04 +0200, Hrvoje Popovski wrote: > > >> Hi all, > > >> > > >> OpenBSD box acts as transit router for /8 networks without pf and with > > >> this sysctls > > >> > > >> ddb.console=1 > > >> kern.pool_debug=0 > > >> net.inet.ip.forwarding=1 > > >> net.inet.ip.ifq.maxlen=8192 > > >> > > >> netstat > > >> 11/8 192.168.11.2 UGS 0 114466419 - 8 > > >> ix0 > > >> 12/8 192.168.12.2 UGS 0 0 - 8 > > >> ix1 > > >> 13/8 192.168.13.2 UGS 0 0 - 8 > > >> myx0 > > >> 14/8 192.168.14.2 UGS 0 0 - 8 > > >> myx1 > > >> 15/8 192.168.15.2 UGS 0 0 - 8 > > >> em3 > > >> 16/8 192.168.16.2 UGS 0 89907239 - 8 > > >> em2 > > >> 17/8 192.168.17.2 UGS 0 65791508 - 8 > > >> bge0 > > >> 18/8 192.168.18.2 UGS 0 0 - 8 > > >> bge1 > > >> > > >> while testing dlg@ "mcl2k2 mbuf clusters" patch with todays -current i > > >> saw that performance with plain -current drops for about 300Kpps vs > > >> -current from 06.10.2016. by bisecting cvs tree it seems that this > > >> commit is guilty for this > > >> > > >> http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/net/if_ethersubr.c?rev=1.240&content-type=text/x-cvsweb-markup > > >> > > > > > > I don't see how this change can affect performance in such a way > > > unless you're sending jumbo packets, but then the packet rates > > > are too high. Are you 100% sure it's this particular change? > > > > > > > No, no, i'm not 100% sure. I was doing this to try to find bottleneck: > > > > cvs -q checkout -D "2016-10-XX" -P src > > > > 2016-10-06 - 900kpps > > 2016-10-07 - 900kpps > > 2016-10-10 - 900kpps > > 2016-10-11 - 650kpps > > 2016-10-11 with if_ethersubr.c 1.239 - 900kpps > > ... > > 2016-10-14 - 650kpps > > 2016-10-14 with dlg@ patch - 900kpps > > 2016-10-14 with dlg@ patch and with if_ethersubr.c 1.239 - 880kpps > > .... > > 2016-10-24 - results are in mail ... > > > > and then i looked at networking diffs from 2016-10-10 and 2016-10-11 and > > it seems that if_ethersubr.c is guilty > > > > tests was done over ix only ... > > > > although as you can see with today's plain -current i'm getting 690kpps > > and with today's -current with if_ethersubr.c 1.239 i'm getting 910kpps > > > > so i thought that there must be something with if_ethersubr.c > > > > I see. I will double check this tomorrow but your approach > looks solid.
it's obviously an interaction between intel nics that do not align their ethernet headers correctly, and the M_PREPEND which i just changed (and you oked) in src/sys/net/if_ethersubr.c r1.240). basically the stack strips the 6 byte ethernet header before pushing the packet into the ip stack, and forwarding causes it to be output as an ethernet packet. the M_PREPEND of 8 bytes in ether_output causes an mbuf to be prefixed cos the frame has 6 bytes free, not 8. the good news is that at least the prepended mbuf gets its ethernet header correctly aligned. dlg
