Bridge performance with vlans on ix(4) and protected ports ) diagnostic tips request

[Tom Smyth]

Hello,

I'm running 6.6 Stable amd64  on an 4 Core VM with ix(4) PCI-E SRIOV
attached Nics
Im seeing performance of about 450Mb/s throughput

it bridges /aggregates about 90 vlans into one physical 10G port  (and
isolates them (with all the vlans in the bridge and in the same
protection
im seeing alot of of packets dropped because of destination
unreachable  ( about 3K destinations unreachable per second)

my guess is that there are bursts of packets coming through and the
cpu cant process them in time... Im wondering is there a handy way of
increasing buffers to smooth out these micro bursts ? to see if this
will improve the situation

with the system running about 20 hours the following stats are

# netstat -nI bridge101 ;netstat -nI ix0;netstat -nI ix1;netstat -nI ix2
NameMtu   Network Address  Ipkts IfailOpkts Ofail Colls
bridge101 1500  3795904105 0
3933822265 44817 0
NameMtu   Network Address  Ipkts IfailOpkts Ofail Colls
ix0 170000:12:c0:88:07:b8 2436105363 0
1377058248 44999 0
NameMtu   Network Address  Ipkts IfailOpkts Ofail Colls
ix1 170800:12:c0:88:07:b9 297914355 0 377157134
570713 0
NameMtu   Network Address  Ipkts IfailOpkts Ofail Colls
ix2 170800:12:c0:88:07:b6 1233726645 0
2173231651 5807022 0

netstat of a random vlans in the bridge

 netstat -nI vlan4001
NameMtu   Network Address  Ipkts IfailOpkts Ofail Colls
vlan4001 170000:12:c0:88:07:b6  1690797 0  3932075 23385 0
ngabr# netstat -nI vlan4002
NameMtu   Network Address  Ipkts IfailOpkts Ofail Colls
vlan4002 170000:12:c0:88:07:b6  5222477 0  8170507 31109 0
ngabr# netstat -nI vlan4003
NameMtu   Network Address  Ipkts IfailOpkts Ofail Colls
vlan4003 170000:12:c0:88:07:b6  7333418 0  7842635 23394 0



I found this using the following command to test
netstat -sr;sleep 5;netstat -sr
routing:
0 bad routing redirects
0 dynamically created routes
0 new gateways due to redirects
128772097 destinations found unreachable
0 uses of a wildcard route
routing:
0 bad routing redirects
0 dynamically created routes
0 new gateways due to redirects
128787229 destinations found unreachable
0 uses of a wildcard route

netstat


output of top command

load averages:  0.81,  0.76,  0.82

ngabr 23:57:04
50 processes: 46 idle, 4 on processor

  up 21:45
CPU0 states:  0.0% user,  0.0% nice,  0.8% sys,  0.4% spin,  8.8%
intr, 90.0% idle
CPU1 states:  0.0% user,  0.0% nice, 34.0% sys,  0.8% spin,  0.0%
intr, 65.2% idle
CPU2 states:  0.0% user,  0.0% nice, 40.2% sys,  1.4% spin,  0.0%
intr, 58.4% idle
CPU3 states:  0.0% user,  0.0% nice,  0.0% sys,  0.0% spin,  0.0%
intr,  100% idle
Memory: Real: 25M/916M act/tot Free: 1050M Cache: 493M Swap: 0K/0K

  PID USERNAME PRI NICE  SIZE   RES STATE WAIT  TIMECPU COMMAND
34304 root  10  -200K 1184K onproc/1  -16.8H 74.22% softnet
77304 root  280 2000K 3940K onproc/2  - 0:00  0.05% top
70327 root -2200K 1184K onproc/3  -21.7H  0.00% idle3
82993 root -2200K 1184K onproc/0  -19.6H  0.00% idle0
93492 root -2200K 1184K sleep/2   -   868:12  0.00% idle2
65795 root -2200K 1184K sleep/1   -   845:29  0.00% idle1
34062 root -2200K 1184K sleep/0   bored12:00  0.00% softclock
67470 _pflogd40  888K  572K sleep/2   bpf   2:34  0.00% pflogd
16631 root  1800K 1184K sleep/2   syncer1:07  0.00% update
37166 root  1000K 1184K sleep/1   bored 1:01  0.00% systqmp
64060 fireman20 1404K 2976K sleep/0   select0:47  0.00% sshd
62514 root   20  696K 1284K sleep/1   poll  0:11  0.00% cron
93672 _ntp   2  -20 1244K 2740K idle  poll  0:09  0.00% ntpd
0 root -1800K 1184K sleep/0   schedul   0:07  0.00% swapper
74711 _syslogd   20 1164K 1512K idle  kqread0:07  0.00% syslogd
 2383 root  68   200K 1184K sleep/2   pgzero0:05  0.00% zerothread
85602 root  180 1052K  908K sleep/3   pause 0:04  0.00% ksh
58290 root -1800K 1184K idle  reaper0:04  0.00% reaper
32404 _dhcp  20  896K  556K idle  poll  0:03  0.00% dhclient
62783 root  1000K 1184K idle  bored 0:01  0.00% systq

dmesg below
OpenBSD 6.6 (GENERIC.MP) #8: Fri Apr 17 15:06:32 MDT 2020

r...@syspatch-66-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 2130526208 (2031MB)
avail mem = 2053304320 (1958MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5920 (9 entries)
bios0: vendor SeaBIOS version

Re: Optimizing pf.conf

[Peter Nicolai Mathias Hansteen]

> 6. mai 2020 kl. 22:00 skrev Lars Bonnesen :
> 
> Is it no longer important to group block/pass in/out for speed optimization?
> 
> I see many "modern" pf.conf where everything is mixed more or less randomly

My advice would be to write your pf.conf in a way that makes sense in your 
environment and is readable to whoever gets to maintain the thing.

As Theo pointed out, the built-in optimizer will do reordering and other tricks 
for performance if there is a need and you do not explicitly disable 
optimization.

All the best,

—
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
"Remember to set the evil bit on all malicious network traffic"
delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.






signature.asc
Description: Message signed with OpenPGP

Re: wireguard on i386

[Aisha Tammy]

On 5/6/20 12:22 PM, Stuart Henderson wrote:
> On 2020-05-06, infoomatic  wrote:
>> Hi,
>>
>> I realized wireguard is not available as binary package for i386. Since
>> this is my only 32bit machine I would setup 32bit VM to build the
>> package. Is it possible to compile it from ports for 32bit? (or is the
>> missing package a sign that it's not available for 32bit architecture?)
> 
> Use wiresep, wireguard-go does not build on i386 (but even on archs
> where both are available, wiresep has several advantages).
> 
> 
Ooof, seems like the change for removing ONLY_FOR_ARCHS had to be rolled back.
Very unfortunate :(

Re: Optimizing pf.conf

[Theo de Raadt]

pfctl has an ruleset optimizer built in, which handles most of that.

So, it is best if you write rules in a way that makes sense.


Lars Bonnesen  wrote:

> Is it no longer important to group block/pass in/out for speed optimization?
> 
> I see many "modern" pf.conf where everything is mixed more or less randomly
> 
> Regards, Lars.

Optimizing pf.conf

[Lars Bonnesen]

Is it no longer important to group block/pass in/out for speed optimization?

I see many "modern" pf.conf where everything is mixed more or less randomly

Regards, Lars.

Re: wireguard on i386

[Aisha Tammy]

On 5/6/20 9:58 AM, infoomatic wrote:
> Hi,
> 
> I realized wireguard is not available as binary package for i386. Since
> this is my only 32bit machine I would setup 32bit VM to build the
> package.
There are two packages wireguard-tools and wireguard-go

Both have been recently updated to work on all platforms, if you are running
-current you should have them available. I don't think they have been backported
to 6.6

 Is it possible to compile it from ports for 32bit? (or is the
> missing package a sign that it's not available for 32bit architecture?)
> 
Yes, both of them can be compiled manually, take a look at the Makefile[1]
to see what the build time dependencies are
> thanks,
> 
> infoomatic
> 

[1]https://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/net/wireguard-go/

support

[Chris Petrik]

0
C USA
P Mississippi
T Gulfport
Z 39501
O Petrik Consulting
I Chris Petrik
A 1610 Thornton, Ave
M ch...@cpettington.com
U http://www.cpettington.com/
B 2282650091
X
N BSD based consulting in the Mississippi area. We specialize in using
OpenBSD as our base go-to Operating System for all services requested.

Re: one-character expansion in shell

[Ottavio Caruso]

On Wed, 6 May 2020 at 14:25, Rudolf Sykora  wrote:
>
> Hello list,
>
>
> is this an expected behaviour?
>
> odin$ ls v?k*
> ls: v?k*: No such file or directory
> odin$ ls v??k*
> výkres.1.pdfvýkres.2.pdfvýkres.5.pdfvýkres.8.pdf
> výkres.10.pdf   výkres.3.pdfvýkres.6.pdfvýkres.9.pdf
> výkres.11.pdf   výkres.4.pdfvýkres.7.pdf
> odin$ locale
> LANG=
> LC_COLLATE="C"
> LC_CTYPE=en_US.UTF-8
> LC_MONETARY="C"
> LC_NUMERIC="C"
> LC_TIME="C"
> LC_MESSAGES="C"
> LC_ALL=
>
>
> It seems I have to use double '??' to mach a single character.
>
> Thanks for comments!

Weird. I can reproduce this on OpenBSD 6.6 + ksh:

oc@OpenBSD:~/tt$ ls
výkres.1.pdfvýkres.2.pdfvýkres.4.pdfvýkres.6.pdfvýkres.8.pdf
výkres.10.pdf   výkres.3.pdfvýkres.5.pdfvýkres.7.pdfvýkres.9.pdf
oc@OpenBSD:~/tt$  ls v?k*
ls: v?k*: No such file or directory
oc@OpenBSD:~/tt$ locale
LANG=en_GB.UTF-8
LC_COLLATE="en_GB.UTF-8"
LC_CTYPE="en_GB.UTF-8"
LC_MONETARY="en_GB.UTF-8"
LC_NUMERIC="en_GB.UTF-8"
LC_TIME="en_GB.UTF-8"
LC_MESSAGES="en_GB.UTF-8"
LC_ALL=


But it works on Debian Stretch + bash:

oc@e130:~/t$ ls
výkres.10.pdf  výkres.2.pdf  výkres.5.pdf  výkres.8.pdf
výkres.11.pdf  výkres.3.pdf  výkres.6.pdf  výkres.9.pdf
výkres.1.pdf   výkres.4.pdf  výkres.7.pdf
oc@e130:~/t$ ls v?k*
výkres.10.pdf  výkres.2.pdf  výkres.5.pdf  výkres.8.pdf
výkres.11.pdf  výkres.3.pdf  výkres.6.pdf  výkres.9.pdf
výkres.1.pdf   výkres.4.pdf  výkres.7.pdf
oc@e130:~/t$ locale
LANG=en_GB.UTF-8
LANGUAGE=
LC_CTYPE="en_GB.UTF-8"
LC_NUMERIC="en_GB.UTF-8"
LC_TIME="en_GB.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY="en_GB.UTF-8"
LC_MESSAGES="en_GB.UTF-8"
LC_PAPER="en_GB.UTF-8"
LC_NAME="en_GB.UTF-8"
LC_ADDRESS="en_GB.UTF-8"
LC_TELEPHONE="en_GB.UTF-8"
LC_MEASUREMENT="en_GB.UTF-8"
LC_IDENTIFICATION="en_GB.UTF-8"
LC_ALL=

-- 
Ottavio Caruso

Re: 10Gbps X520 network adapter only passing 3.5Gbps

[Jordan Geoghegan]

On 2020-05-06 04:04, Stuart Henderson wrote:

On 2020-05-06, Jordan Geoghegan  wrote:


On 2020-05-04 06:42, Kalle Kadakas wrote:

Greetings OpenBSD community,

I am running into severe bandwidth limitations whilst passing traffic
   through an OpenBSD firewall.
The NIC in use is an Intel 10Gb 2-port X520 adapter from which I would
   hope to pass through at least 7Gbps+, yet the best results I have
   gotten is only around 3.5Gbps.

The results of bandwidth measurements (iperf for 30sec...
   

As has been discussed on misc previously, iperf is not suitable for
benchmarking networking throughput on OpenBSD. It ends up just
benchmarking the gettimeofday syscall (something that is cheap on Linux,
but relatively expensive on OpenBSD I'm told).

clock_gettime. It's iperf3 that calls this often; iperf not so much.
But when testing with default options, you may see higher numbers from
iperf3: the direct comparison isn't fair though because it uses 128K
TCP buffers by default, whereas iperf uses the OS default.

On Linux clock_gettime often doesn't use a system call, on OpenBSD it
does (and with some of the mitigations for cpu bugs, system calls are
more expensive than they used to be).


For best results, use tcpbench for your OpenBSD networking benchmarks.

For accurate results of packet forwarding performance, use fast packet
sources/sinks running whatever OS either side of an OpenBSD router.




Thanks for clarifying Stuart, I knew I was about 70% of the way there.

Jordan

Re: wireguard on i386

[Stuart Henderson]

On 2020-05-06, infoomatic  wrote:
> Hi,
>
> I realized wireguard is not available as binary package for i386. Since
> this is my only 32bit machine I would setup 32bit VM to build the
> package. Is it possible to compile it from ports for 32bit? (or is the
> missing package a sign that it's not available for 32bit architecture?)

Use wiresep, wireguard-go does not build on i386 (but even on archs
where both are available, wiresep has several advantages).

Re: OSPF lsa_check issue

[Claudio Jeker]

On Wed, May 06, 2020 at 03:23:06PM +0100, Richard Chivers wrote:
> Hi,
> 
> Thanks so much for the diff, it appears to have resolved the issue.
> 
> We are now trying to establish whether we need the fix widely deployed or
> only on the box that originates with the large LSA updates, pushing it over
> the 1500mtu.
> 
> We are going to run some tests, but our expectation is that when the DR
> sends the message from the originating router on to its neighbors that they
> will then see the same issue.
> 
> Out of interest is there any way of just announcing a single network.
> 
> In this particular case the large LS-Update is caused because we have many
> interfaces, but these are all carp so will failover in one hit anyway. We
> have allocated 10.128.0.0/16 to this firewall so there are many networks,
> but anything in our network with a destination of 10.128.0.0/16 can end up
> here.
> 
> We tried something like *redistribute 10.128.0.0/16 
> depend on carp0*, but what that appears to do is limit advertisements to
> the subnets that fall within that range, so we still have a very large LSA
> update anyway.
> 
> Just wondering if there was any workaround, as it would just simplify
> processing etc.
> 
> It is probably a non issue anyway now, with the fix, but just interested if
> anyone has done anything similar.

Without the exact config it is hard to judge but you are advertising a lot
of stub networks in the router lsa. stub networks are from interface rules
that are passive or have no active peers. So to reduce the size of the
router LSA an option is to remove some of the interfaces and change them
to redistribute connected which uses Type-5 LSA instead.

-- 
:wq Claudio

Re: OSPF lsa_check issue

[Richard Chivers]

Hi,

Thanks so much for the diff, it appears to have resolved the issue.

We are now trying to establish whether we need the fix widely deployed or
only on the box that originates with the large LSA updates, pushing it over
the 1500mtu.

We are going to run some tests, but our expectation is that when the DR
sends the message from the originating router on to its neighbors that they
will then see the same issue.

Out of interest is there any way of just announcing a single network.

In this particular case the large LS-Update is caused because we have many
interfaces, but these are all carp so will failover in one hit anyway. We
have allocated 10.128.0.0/16 to this firewall so there are many networks,
but anything in our network with a destination of 10.128.0.0/16 can end up
here.

We tried something like *redistribute 10.128.0.0/16 
depend on carp0*, but what that appears to do is limit advertisements to
the subnets that fall within that range, so we still have a very large LSA
update anyway.

Just wondering if there was any workaround, as it would just simplify
processing etc.

It is probably a non issue anyway now, with the fix, but just interested if
anyone has done anything similar.

Regards

Richard



On Wed, May 6, 2020 at 9:52 AM Claudio Jeker 
wrote:

> On Wed, May 06, 2020 at 09:33:11AM +0100, Richard Chivers wrote:
> > Hi,
> >
> > Some progress has been made, we can now replicate this consistently and
> it
> > appears that whenever a LS update exceeds the mtu (1500) we get this
> issue
> > of lsa_check bad age.
> >
> > When running with the diff Claudio sent we start getting a bunch of
> errors
> > complaining about:
> >
> > recv_ls_update: bad packet size, neighbor ID x.x.x.x
> > lsa_check: bad packet size
> >
> > We don't ever move to a state of FULL/DR or similar.
> >
> > Does anyone have any suggestions? We are just starting to look at the
> wider
> > code to see if we can comprehend what may be occurring, but it will
> > likely be a steep learning curve :)
> >
>
> Just realized that my diff was wrong since ibuf_reserve() would change the
> write position of the buffer and so you end up with some empty space in
> the buffer.
>
> Here is a better diff. This is using ibuf_size to get the current write
> position and then ibuf_seek() to write the age back into the right spot.
> Using the position instead of the pointer has the benefit that a realloc()
> in ibuf_add() will not result in the stale pointer to lsage that the
> current code has.
>
> I have currently no ospf setup so my testing is limited.
> --
> :wq Claudio
>
> Index: lsupdate.c
> ===
> RCS file: /cvs/src/usr.sbin/ospfd/lsupdate.c,v
> retrieving revision 1.47
> diff -u -p -r1.47 lsupdate.c
> --- lsupdate.c  19 Nov 2019 09:55:55 -  1.47
> +++ lsupdate.c  6 May 2020 08:48:19 -
> @@ -175,8 +175,8 @@ int
>  add_ls_update(struct ibuf *buf, struct iface *iface, void *data,
> u_int16_t len,
>  u_int16_t older)
>  {
> -   void*lsage;
> -   u_int16_tage;
> +   size_t  ageoff;
> +   u_int16_t   age;
>
> if ((size_t)iface->mtu < sizeof(struct ip) + sizeof(struct
> ospf_hdr) +
> sizeof(u_int32_t) + ibuf_size(buf) + len + MD5_DIGEST_LENGTH) {
> @@ -186,7 +186,7 @@ add_ls_update(struct ibuf *buf, struct i
> return (0);
> }
>
> -   lsage = ibuf_reserve(buf, 0);
> +   ageoff = ibuf_size(buf);
> if (ibuf_add(buf, data, len)) {
> log_warn("add_ls_update");
> return (0);
> @@ -198,7 +198,7 @@ add_ls_update(struct ibuf *buf, struct i
> if ((age += older + iface->transmit_delay) >= MAX_AGE)
> age = MAX_AGE;
> age = htons(age);
> -   memcpy(lsage, , sizeof(age));
> +   memcpy(ibuf_seek(buf, ageoff, sizeof(age)), , sizeof(age));
>
> return (1);
>  }
>

Re: one-character expansion in shell

[Philipp Buehler]

Am 06.05.2020 15:54 schrieb Ingo Schwarze:

Your misunderstandiing is that file names consist of characters.
They do not.  They consist of bytes, and to match two bytes,
you need two question marks.


One can hold for the OP; the ksh(1) manpage talks about
"characters" in 'File name patterns' throughout.

Just two bytes ;-)
--
pb

wireguard on i386

[infoomatic]

Hi,

I realized wireguard is not available as binary package for i386. Since
this is my only 32bit machine I would setup 32bit VM to build the
package. Is it possible to compile it from ports for 32bit? (or is the
missing package a sign that it's not available for 32bit architecture?)

thanks,

infoomatic

Re: one-character expansion in shell

[Ingo Schwarze]

Hi Rudolf,

Rudolf Sykora wrote on Wed, May 06, 2020 at 03:25:09PM +0200:

> is this an expected behaviour?

Yes, that is expected behaviour.

> odin$ ls v?k*
> ls: v?k*: No such file or directory
> odin$ ls v??k*

[ some file names containing two-byte UTF-8 sequences ]

> odin$ locale

The locale is totally irrelevant with respect to file names.
The locale is a user preference.  Different users can set different
locales.  Even the same user can set different locales for different
instances of programs they are running.

By contrast, names of files are system-wide properties.
They are necessarily the same for all users, no matter the locale.

> It seems I have to use double '??' to mach a single character.

Your misunderstandiing is that file names consist of characters.
They do not.  They consist of bytes, and to match two bytes,
you need two question marks.

You can use all kinds of bytes in file names, there are very few
restrictions: e.g. you cannot use a slash in a file name, and you
cannot manually create a file called "." or "..".

However, it is best practice to only use printable non-whitespace
ASCII bytes in file names.  It is well-known that using whitespace
characters in file names poses notorious security hazards.  Using
non-printable or non-ASCII bytes usually causes confusion, so it
certainly isn't recommended either.

That said, people sometimes have to deal with files named by other,
careless or reckless people, so OpenBSD tries to handle file names
in a best-effort manner when they contain unusual bytes.  For
example, when a file name contains byte sequences that can be
interpreted as UTF-8, ls(1) in xterm(1) displays these byte sequences
with the corresponding Unicode glyphs if you have set a UTF-8 locale.
But that doesn't imply filenames suddenly use UTF-8 in any sense.

Yours,
  Ingo

one-character expansion in shell

[Rudolf Sykora]

Hello list,


is this an expected behaviour?

odin$ ls v?k*
ls: v?k*: No such file or directory
odin$ ls v??k*
výkres.1.pdfvýkres.2.pdfvýkres.5.pdfvýkres.8.pdf
výkres.10.pdf   výkres.3.pdfvýkres.6.pdfvýkres.9.pdf
výkres.11.pdf   výkres.4.pdfvýkres.7.pdf
odin$ locale
LANG=
LC_COLLATE="C"
LC_CTYPE=en_US.UTF-8
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_MESSAGES="C"
LC_ALL=


It seems I have to use double '??' to mach a single character.

Thanks for comments!


Best regards
Rudolf Sykora

Re: SpeedTest-cli results accuracy ?

[Stuart Henderson]

On 2020-05-06, Judah Kocher  wrote:
> speedtest-cli is horribly inaccurate in my experience. I used it when I 
> first started using OpenBSD as a router and spent mor etime than I care 
> to admit "troubleshooting" before realizing I was getting the correct 
> speeds on devices on the network.

Apart from anything else it's only as good as the speedtest server,
and for people on faster connections they often can't cope.

Re: 10Gbps X520 network adapter only passing 3.5Gbps

[Stuart Henderson]

On 2020-05-06, Jordan Geoghegan  wrote:
>
>
> On 2020-05-04 06:42, Kalle Kadakas wrote:
>> Greetings OpenBSD community,
>>
>> I am running into severe bandwidth limitations whilst passing traffic
>>   through an OpenBSD firewall.
>> The NIC in use is an Intel 10Gb 2-port X520 adapter from which I would
>>   hope to pass through at least 7Gbps+, yet the best results I have
>>   gotten is only around 3.5Gbps.
>>
>> The results of bandwidth measurements (iperf for 30sec...
>>   
>
> As has been discussed on misc previously, iperf is not suitable for 
> benchmarking networking throughput on OpenBSD. It ends up just 
> benchmarking the gettimeofday syscall (something that is cheap on Linux, 
> but relatively expensive on OpenBSD I'm told).

clock_gettime. It's iperf3 that calls this often; iperf not so much.
But when testing with default options, you may see higher numbers from
iperf3: the direct comparison isn't fair though because it uses 128K
TCP buffers by default, whereas iperf uses the OS default.

On Linux clock_gettime often doesn't use a system call, on OpenBSD it
does (and with some of the mitigations for cpu bugs, system calls are
more expensive than they used to be).

> For best results, use tcpbench for your OpenBSD networking benchmarks.

For accurate results of packet forwarding performance, use fast packet
sources/sinks running whatever OS either side of an OpenBSD router.

Re: 10Gbps X520 network adapter only passing 3.5Gbps

[Kalle Kadakas]

Thank you Jordan for looking into this!
I ran the test with tcpbench for 2min (pf+isakmpd+lacp) and the
results were pretty much the same, a bit better:
Peak Mbps: 3588.227 Avg Mbps: 3533.040

Cannot really run all 4 tests right away as this is a production node
and I currently do not have any alternative HW to set it up just yet,
but I am planning on doing the full round of testing with different
server hardware in about a months time when I get the hardware.

Also to note is that when I had similar load in a real-life situation,
(some database witchcraft uses as much bandwidth as it can get) then
when the production node reached ~3.5Gbps, there really was no
bandwidth left to serve the essential services like relayd checks.
This is what initially led me to this rabbithole of testing the
bandwidth and trying to figure out the bottleneck. :)

Best regards,
Kalle

On Wed, May 6, 2020 at 10:41 AM Jordan Geoghegan 
wrote:

>
>
> On 2020-05-04 06:42, Kalle Kadakas wrote:
> > Greetings OpenBSD community,
> >
> > I am running into severe bandwidth limitations whilst passing traffic
> >   through an OpenBSD firewall.
> > The NIC in use is an Intel 10Gb 2-port X520 adapter from which I would
> >   hope to pass through at least 7Gbps+, yet the best results I have
> >   gotten is only around 3.5Gbps.
> >
> > The results of bandwidth measurements (iperf for 30sec...
> >
>
> As has been discussed on misc previously, iperf is not suitable for
> benchmarking networking throughput on OpenBSD. It ends up just
> benchmarking the gettimeofday syscall (something that is cheap on Linux,
> but relatively expensive on OpenBSD I'm told). For best results, use
> tcpbench for your OpenBSD networking benchmarks.
>
>
>

macppc wsconsctl screen brightness

[rgc]

misc@

i've been following 67-beta ... and just noticed this since #711.
it is possible this issue occured earlier.
i have no idea if display.brightness worked on 6.6-stable

macppc.html shows i can do this via "wsconsctl -w XXX".
** note that wsconsctl doesn't have a "-w" option so macppc.html might need
to be updated **

brightness setting still works via ADB keyboard on my PowerBook6,7 (iBook G4)
but setting display.brightness from command line does nothing on this device
and on another device (PowerBook G4). both are radeondrm

rgc:/home/rgc/openbsd/www:29$ sysctl kern.version
kern.version=OpenBSD 6.7 (GENERIC) #712: Tue May  5 10:40:23 MDT 2020
dera...@macppc.openbsd.org:/usr/src/sys/arch/macppc/compile/GENERIC

rgc:/home/rgc/openbsd/www:26$ doas wsconsctl display.brightness=50
rgc:/home/rgc/openbsd/www:27$ doas wsconsctl -a
keyboard.type=usb
keyboard.bell.pitch=400
keyboard.bell.period=100
keyboard.bell.volume=50
keyboard.bell.pitch.default=400
keyboard.bell.period.default=100
keyboard.bell.volume.default=50
wsconsctl: Use explicit arg to view keyboard.map.
keyboard.repeat.del1=400
keyboard.repeat.deln=100
keyboard.repeat.del1.default=400
keyboard.repeat.deln.default=100
keyboard.ledstate=0
keyboard.encoding=us.swapctrlcaps
keyboard1.type=usb
keyboard1.bell.pitch=400
keyboard1.bell.period=100
keyboard1.bell.volume=50
keyboard1.bell.pitch.default=400
keyboard1.bell.period.default=100
keyboard1.bell.volume.default=50
wsconsctl: Use explicit arg to view keyboard1.map.
keyboard1.repeat.del1=400
keyboard1.repeat.deln=100
keyboard1.repeat.del1.default=400
keyboard1.repeat.deln.default=100
keyboard1.ledstate=0
keyboard1.encoding=us.swapctrlcaps
mouse.type=usb
mouse.rawmode=1
mouse.scale=0,0,0,0,0,0,0
mouse.reverse_scrolling=0
mouse1.type=usb
mouse1.reverse_scrolling=0
display.type=radeondrm
display.width=1440
display.height=960
display.depth=32
display.fontwidth=12
display.fontheight=24
display.emulations=vt100
display.screentypes=std
display.focus=4
display.screen_on=250
display.screen_off=6
display.vblank=on
display.kbdact=on
display.msact=off
display.outact=on


i have an 67-beta i386 machine where wsconsctl work correctly

rgc:/home/rgc:8$ doas wsconsctl display.brightness=50
display.brightness -> 50.00%


~ rgc

Re: OSPF lsa_check issue

[Claudio Jeker]

On Wed, May 06, 2020 at 09:33:11AM +0100, Richard Chivers wrote:
> Hi,
> 
> Some progress has been made, we can now replicate this consistently and it
> appears that whenever a LS update exceeds the mtu (1500) we get this issue
> of lsa_check bad age.
> 
> When running with the diff Claudio sent we start getting a bunch of errors
> complaining about:
> 
> recv_ls_update: bad packet size, neighbor ID x.x.x.x
> lsa_check: bad packet size
> 
> We don't ever move to a state of FULL/DR or similar.
> 
> Does anyone have any suggestions? We are just starting to look at the wider
> code to see if we can comprehend what may be occurring, but it will
> likely be a steep learning curve :)
> 

Just realized that my diff was wrong since ibuf_reserve() would change the
write position of the buffer and so you end up with some empty space in
the buffer.

Here is a better diff. This is using ibuf_size to get the current write
position and then ibuf_seek() to write the age back into the right spot.
Using the position instead of the pointer has the benefit that a realloc()
in ibuf_add() will not result in the stale pointer to lsage that the
current code has.

I have currently no ospf setup so my testing is limited.
-- 
:wq Claudio

Index: lsupdate.c
===
RCS file: /cvs/src/usr.sbin/ospfd/lsupdate.c,v
retrieving revision 1.47
diff -u -p -r1.47 lsupdate.c
--- lsupdate.c  19 Nov 2019 09:55:55 -  1.47
+++ lsupdate.c  6 May 2020 08:48:19 -
@@ -175,8 +175,8 @@ int
 add_ls_update(struct ibuf *buf, struct iface *iface, void *data, u_int16_t len,
 u_int16_t older)
 {
-   void*lsage;
-   u_int16_tage;
+   size_t  ageoff;
+   u_int16_t   age;
 
if ((size_t)iface->mtu < sizeof(struct ip) + sizeof(struct ospf_hdr) +
sizeof(u_int32_t) + ibuf_size(buf) + len + MD5_DIGEST_LENGTH) {
@@ -186,7 +186,7 @@ add_ls_update(struct ibuf *buf, struct i
return (0);
}
 
-   lsage = ibuf_reserve(buf, 0);
+   ageoff = ibuf_size(buf);
if (ibuf_add(buf, data, len)) {
log_warn("add_ls_update");
return (0);
@@ -198,7 +198,7 @@ add_ls_update(struct ibuf *buf, struct i
if ((age += older + iface->transmit_delay) >= MAX_AGE)
age = MAX_AGE;
age = htons(age);
-   memcpy(lsage, , sizeof(age));
+   memcpy(ibuf_seek(buf, ageoff, sizeof(age)), , sizeof(age));
 
return (1);
 }

Re: OSPF lsa_check issue

[Richard Chivers]

Hi,

Some progress has been made, we can now replicate this consistently and it
appears that whenever a LS update exceeds the mtu (1500) we get this issue
of lsa_check bad age.

When running with the diff Claudio sent we start getting a bunch of errors
complaining about:

recv_ls_update: bad packet size, neighbor ID x.x.x.x
lsa_check: bad packet size

We don't ever move to a state of FULL/DR or similar.

Does anyone have any suggestions? We are just starting to look at the wider
code to see if we can comprehend what may be occurring, but it will
likely be a steep learning curve :)

Thanks

Richard



On Tue, May 5, 2020 at 1:04 PM Richard Chivers 
wrote:

> Hi,
>
> We have sent the pcap directly for the raw packets.
>
> In terms of the above change, we haven't compiled ospf previously, we will
> give it a go and see how we get on.
>
> Are we ok to clone off the github mirror?
>
> Cheers
>
> Richard
>
> On Tue, May 5, 2020 at 10:22 AM Claudio Jeker 
> wrote:
>
>> On Tue, May 05, 2020 at 10:51:40AM +0200, Claudio Jeker wrote:
>> > On Tue, May 05, 2020 at 09:07:34AM +0100, Richard Chivers wrote:
>> > > After some more work this morning we have managed to extract the
>> > > information from tcpdump of the full LS-Update packet, we couldn't
>> see it
>> > > on bsd, but running:
>> > >
>> > > tcpdump -v -r ~/Downloads/ospf.pcap on osx did the trick.
>> > >
>> > > What we are seeing is that a pair of firewalls are both sending
>> updates
>> > > like this:
>> > >
>> > > 07:16:09.346525 IP (tos 0xc0, ttl 1, id 47473, offset 0, flags [+],
>> proto
>> > > OSPF (89), length 1500)
>> > > x.x.x.x > ospf-dsig.mcast.net: OSPFv2, LS-Update, length 1480
>> [len 1672]
>> > > Router-ID x.x.x.x, Backbone Area, Authentication Type: simple (1)
>> > > Simple text password: dslkfjld, 1 LSA
>> > >  LSA #1
>> > >  Advertising Router x.x.x.x, seq 0x806e, age 0s, length 1624
>> > >Router LSA (1), LSA-ID: x.x.x.x
>> > >Options: [External]
>> > >Router LSA Options: [ASBR]
>> > >  Stub Network: 10.128.32.128, Mask: 255.255.255.128
>> > > topology default (0), metric 10
>> > >  Stub Network: 10.128.9.0, Mask: 255.255.255.128
>> > > *{ another 50 or so networks here}*
>> > >
>> > > Each time we get one of these updates the DR logs the lsa_check: bad
>> age.
>> > >
>> > > Another 5 or so seconds later the same LS-Update comes in with the
>> same seq
>> > > number. This appears to continue indefinitely. Our only fix appears
>> to be
>> > > restarting ospfd on the routers.
>> > >
>> > > Does anyone have an idea what is going wrong here?
>> > >
>> > > Something we have considered being a problem is that we do have many
>> > > interfaces, we have 90 or so, so the LS-Update packets are quite
>> large and
>> > > do get fragmented, as we are using a 1500mtu.
>> > >
>> > > The fact that ospfd sees the age and complains though makes us think
>> this
>> > > is not a problem.
>> > >
>> >
>> > Looking at the tcpdump output there is something strange with the
>> various
>> > reported length fields. Is it possible to get the raw packet dumps?
>> >
>>
>> Can you try the following diff and see if it fixes the issue?
>>
>> --
>> :wq Claudio
>>
>> Index: lsupdate.c
>> ===
>> RCS file: /cvs/src/usr.sbin/ospfd/lsupdate.c,v
>> retrieving revision 1.47
>> diff -u -p -r1.47 lsupdate.c
>> --- lsupdate.c  19 Nov 2019 09:55:55 -  1.47
>> +++ lsupdate.c  5 May 2020 09:20:50 -
>> @@ -186,7 +186,7 @@ add_ls_update(struct ibuf *buf, struct i
>> return (0);
>> }
>>
>> -   lsage = ibuf_reserve(buf, 0);
>> +   lsage = ibuf_reserve(buf, len);
>> if (ibuf_add(buf, data, len)) {
>> log_warn("add_ls_update");
>> return (0);
>>
>

Re: 10Gbps X520 network adapter only passing 3.5Gbps

[Jordan Geoghegan]

On 2020-05-04 06:42, Kalle Kadakas wrote:

Greetings OpenBSD community,

I am running into severe bandwidth limitations whilst passing traffic
  through an OpenBSD firewall.
The NIC in use is an Intel 10Gb 2-port X520 adapter from which I would
  hope to pass through at least 7Gbps+, yet the best results I have
  gotten is only around 3.5Gbps.

The results of bandwidth measurements (iperf for 30sec...
  


As has been discussed on misc previously, iperf is not suitable for 
benchmarking networking throughput on OpenBSD. It ends up just 
benchmarking the gettimeofday syscall (something that is cheap on Linux, 
but relatively expensive on OpenBSD I'm told). For best results, use 
tcpbench for your OpenBSD networking benchmarks.