> On Jan 19, 2020, at 12:01 PM, Greg Troxel <g...@lexort.com> wrote: > > Chavdar Ivanov <ci4...@gmail.com> writes: > >>> It looks like you are using vlan support on Y. Try without also. >> >> That may be something to look at. This is my NVMM host as well, every >> boot I recreate tap[0..5] for use by the NVMM guests (but the tests >> were done without any of them running). >> >> I am not using vlans deliberately - the switch upstairs is a dumb one, >> although the one downstaris is managed and has (unusued at the moment) >> vlan support. The interfaces are created simply with /etc/ifconfig.wm0 >> - just 'inet 192.168.0.29 netmask 255.255.255.0 up description "My >> LAN"' and /etc/ifconfig.bridge0 - > > I meant that hardware vlan support was enabled. I really doubt this is > the issue, but it seems easy to try the easy things. > > Also, I forgot my other usual advice. > > $ netstat -s > BEFORE > $ # do iperf > $ netstat -s > AFTER > $ diff -u BEFORE AFTER > > to find out which counters that you didn't even know existed are > changing in perhaps interesting ways. > >> From the XCP-NG host to the NetBSD laptop: >> >> $ iperf3 -c ymir.lorien.lan >> Connecting to host ymir.lorien.lan, port 5201 >> [ 4] local 192.168.0.5 port 36036 connected to 192.168.0.29 port 5201 >> [ ID] Interval Transfer Bandwidth Retr Cwnd >> [ 4] 0.00-1.00 sec 45.9 MBytes 385 Mbits/sec 0 66.5 KBytes >> [ 4] 1.00-2.00 sec 64.2 MBytes 539 Mbits/sec 0 100 KBytes >> [ 4] 2.00-3.00 sec 81.3 MBytes 682 Mbits/sec 0 132 KBytes >> [ 4] 3.00-4.00 sec 99.4 MBytes 834 Mbits/sec 0 163 KBytes >> [ 4] 4.00-5.00 sec 109 MBytes 911 Mbits/sec 0 205 KBytes >> [ 4] 5.00-6.00 sec 111 MBytes 928 Mbits/sec 0 205 KBytes >> [ 4] 6.00-7.00 sec 111 MBytes 928 Mbits/sec 0 205 KBytes >> [ 4] 7.00-8.00 sec 111 MBytes 932 Mbits/sec 0 205 KBytes >> [ 4] 8.00-9.00 sec 111 MBytes 930 Mbits/sec 0 205 KBytes >> [ 4] 9.00-10.00 sec 111 MBytes 932 Mbits/sec 0 205 KBytes >> - - - - - - - - - - - - - - - - - - - - - - - - - >> [ ID] Interval Transfer Bandwidth Retr >> [ 4] 0.00-10.00 sec 954 MBytes 800 Mbits/sec 0 sender >> [ 4] 0.00-10.00 sec 953 MBytes 800 Mbits/sec >> receiver >> >> Starts a bit slower, but after the fourth interval reaches along the maximum. > > That's just 1s periods within the same TCP connection. That is more > expected than subsequent TCP connections. But, this is a clue of > something to perhaps change. NetBSD has defaults for send and receive > buffer sizes, and some notion of auto tuning. I am not really clear > where iperf3 is getting those "Cwnd" values, but perhaps it is able to > obtain them from the Linux kernel? > > On some machines I have > > net.inet.tcp.sendspace=131072 > net.inet.tcp.recvspace=131072 > net.inet6.tcp6.sendspace=131072 > net.inet6.tcp6.recvspace=131072 > > net.inet.tcp.recvbuf_auto=0 > net.inet.tcp.sendbuf_auto=0 > net.inet6.tcp6.recvbuf_auto=0 > net.inet6.tcp6.sendbuf_auto=0 > > which may not be the right thing, but at one point I had trouble with > the auto stuff not rampning up fast enough. > >> When the server is the B laptop running W10, I get: >> >> $ iperf3 -c brutus.lorien.lan >> Connecting to host brutus.lorien.lan, port 5201 >> [ 4] local 192.168.0.5 port 43654 connected to 192.168.0.36 port 5201 >> [ ID] Interval Transfer Bandwidth Retr Cwnd >> [ 4] 0.00-1.00 sec 106 MBytes 885 Mbits/sec 0 220 KBytes >> [ 4] 1.00-2.00 sec 108 MBytes 902 Mbits/sec 0 220 KBytes >> [ 4] 2.00-3.00 sec 112 MBytes 938 Mbits/sec 0 220 KBytes >> [ 4] 3.00-4.00 sec 111 MBytes 934 Mbits/sec 0 220 KBytes >> [ 4] 4.00-5.00 sec 112 MBytes 935 Mbits/sec 0 220 KBytes >> [ 4] 5.00-6.00 sec 112 MBytes 941 Mbits/sec 0 220 KBytes >> [ 4] 6.00-7.00 sec 112 MBytes 941 Mbits/sec 0 220 KBytes >> [ 4] 7.00-8.00 sec 109 MBytes 917 Mbits/sec 0 220 KBytes >> [ 4] 8.00-9.00 sec 112 MBytes 943 Mbits/sec 0 220 KBytes >> [ 4] 9.00-10.00 sec 112 MBytes 942 Mbits/sec 0 220 KBytes >> - - - - - - - - - - - - - - - - - - - - - - - - - >> [ ID] Interval Transfer Bandwidth Retr >> [ 4] 0.00-10.00 sec 1.08 GBytes 928 Mbits/sec 0 sender >> [ 4] 0.00-10.00 sec 1.08 GBytes 928 Mbits/sec >> receiver >> >> - e.g. from the start the speed is close to the max. > > close, but the first interval is slower, indicating some rampup. > That's expected. > >> The lack of symetry is strange - from NetBSD to W10 - full speed; from >> W10 to NetBSD - about a third... At the same time there is no >> significant difference if instead of W10 you put Linux or FreeBSD - >> both ways it is similar. And it can't be thrown at iperf3 on W10 only >> - when the server is Linux or FreeBSD, the speed is as expected. > > I wouldn't expect symmetry. A TCP connection being exercised at full > speed involves code on the sender that deals with the congestion window, > fast retransmit, etc. based on acknowledgements (and SACK). The > receiver's code that matters is about generating acks (and there is > usually some delayed ack notion). Similarly, the sending side of one > driver and receiving side of the other driver are being stressed.
Two things: First, Powerline adapters only get 1/7th the advertised speed. They send “1Gigabit” of data, but they’re really sending ~130Mbit of the same data 7 times because of how noisy power cabling is. Second, try adding -l 9000 to your iperf3 tests. The Linux distributions I’ve used have this as the default and I’ve noticed big speed improvements doing this, but I have no clue why it works. Thanks, Jason M. Sent from my iPhone
