On Wed, Jan 22, 2020 at 1:05 PM Lyubomir Yotov <l.yo...@gmail.com> wrote:
> Dear Olivier, > All the best for the New 2020! > In the last few weeks I did some tests with a colleague (a Linux guy) > about 10Gbps performance and BSDRP. The test was between two servers with > connected through a switch. I use Chelsio T520-CR on two HP DL360 with dual > Xeon E5520@2,27GHz. I was a bit surprised that we couldn't get more than > ~3.8Gbps-4.0Bgps with iperf (iperf2 and iperf3 single flow, multiple > flows...), both BSDRP v. 1.96 and stock FreeBSD 12.0. We did the same tests > (BSDRP v. 19.6 only) on two directly connected Dell R610 with X5670@2.90GHz > with the same success. > At the same time with a Proxmox I can get ~7Gbps, which seems a reasonable > result. When using the netmap userland tools I can see that ~14Mpps are > going to the switch. So it is not CPU bound IMHO. > I searched for tunings on the web but what I found was already set in > BSDRP. Any help would be appreciated as I intend to use the cards in > production but the current performance is not satisfactory. > > Hi, BSDRP guide instructs how to tune a FreeBSD for a routing use case: This doesn't suit the case of the server being an end-host (ie: use case of TCP sessions generated from/to the BSDRP). I've tested between on 8 core Atom (C2758 @ 2.40GHz) under BSDRP 1.96 too, one with a 10G Intel and the other with 10G Chelsio: - Command line on the server: iperf3 -s - command line on the client (with 8 threads, because 8 cores: iperf3 -c 198.18.0.1 -P 8 -w 1.5M -l 1M - With TSO/LRO disabled, as instructed Result, with my router-use case tuning (so not good for TCP end host): [SUM] 0.00-10.09 sec 5.45 GBytes 4.64 Gbits/sec 0 sender [SUM] 0.00-10.12 sec 5.45 GBytes 4.63 Gbits/sec receiver => Which is, indeed not good performance (for a TCP server). Let's fix this use-case to optimise it a TCP end host (not the job of a router), just by re-enabling TSO/LRO features on their NIC: [root@server]~# ifconfig ix0 tso lro [root@client]~# ifconfig cxl0 tso lro New result: [SUM] 0.00-10.00 sec 10.9 GBytes 9.32 Gbits/sec 417 sender [SUM] 0.00-10.04 sec 10.8 GBytes 9.28 Gbits/sec receiver => Now the TCP host performance seems correct. Does it "fix" your problem ? Regards, Olivier
_______________________________________________ Bsdrp-users mailing list Bsdrp-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bsdrp-users
