You might want to pull in 5.4-current instead. One you have is not that current any more. :)
On 7 aug 2013, at 16:26, Maxim Khitrov <m...@mxcrypt.com> wrote: > Hi all, > > I'm looking for performance measuring and tuning advice for 10 gigabit > Ethernet. I have a pair of Lanner FW-8865 systems that will be used as > firewalls for the local network. Each one has a Xeon E3-1270v2 CPU, > Intel X540 10GbE NIC (PCIe 3.0 8x), and 8GB DDR3-1600 ECC RAM. Before > putting them into production I wanted to do some throughput testing, > so I connected one directly to the other (via ix0 interfaces) and used > iperf to see how much data I can push through. I also disabled pf for > now, but will do some additional testing with it enabled later on. The > kernel is 5.3 amd64 GENERIC.MP. > > The initial iperf runs couldn't go beyond ~3.2 Gbps: > > # server: iperf -s > # client: iperf -c 192.168.1.3 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.2 sec 3.84 GBytes 3.22 Gbits/sec > > Increasing the TCP window size to 256 KB (seems to be the upper limit) > brings this up to ~4.2 Gbps: > > # server: iperf -s -w 256k > # client: iperf -c 192.168.1.3 -w 256k > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.1 sec 4.96 GBytes 4.22 Gbits/sec > > Increasing the MTU on both ix0 interfaces to 9000 gives me ~7.2 Gbps: > > # server: ifconfig ix0 mtu 9000 && iperf -s -w 256k > # client: ifconfig ix0 mtu 9000 && iperf -c 192.168.1.3 -w 256k -m > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.0 sec 8.39 GBytes 7.21 Gbits/sec > [ 3] MSS size 8948 bytes (MTU 8988 bytes, unknown interface) > > This is where I'm stuck at the moment. When running iperf on > 127.0.0.1, which should only test CPU and memory, I get 11.6 Gbps. > I've read the "Network Tuning and Performance Guide" @ calomel.org, > but none of the tips there help me in getting beyond 7 Gbps on the > physical interfaces. > > I'm also slightly concerned about the performance at the default MTU > of 1500. Looking at `ifconfig ix0 hwfeatures` output (below), it seems > that the ix driver does not support any checksum offloading for the > X540. I wonder if that could be a reason for the poor performance? > > ix0: flags=28843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,NOINET6> mtu 9000 > hwfeatures=30<VLAN_MTU,VLAN_HWTAGGING> hardmtu 16110 > lladdr 00:90:0b:56:12:0c > priority: 0 > groups: LAN SVR > media: Ethernet autoselect (10GbaseT full-duplex) > status: active > inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255 > > Any there any sysctl parameters that I should play with? Any other > system stats that I should monitor? I did a few runs while watching > `top` and `systat vmstat`, but didn't see any problem indications > there. I should also note that I couldn't run iperf in UDP mode - the > client segfaults any time I increase the bandwidth beyond 300 Mbps. No > idea why, but I'm more interested in TCP performance anyway. > > - Max
