On Wed, Feb 5, 2020 at 3:49 PM Lyubomir Yotov <l.yo...@gmail.com> wrote:
> Dear Olivier, > > Hi, > The switch is configured with: > > configure fdb agingtime 0 > disable lldp ports 1,3,5,7 > > disable flow-control tx ports 1,3,5,7 > disable flow-control rx ports 1,3,5,7 > > create vlan vlan2 tag 2 > create vlan vlan3 tag 3 > > configure vlan "Default" del port 1,3,5,7 > configure vlan2 add ports 1,3 > configure vlan3 add ports 5,7 > No static MAC address configured on the switch ? So are you pinging the device between them before starting the benches to be sure the switch have fully populated MAC addresses table ? => If not the switch will "broadcast" packets to all ports belonging to the vlan (with potential huge performance impact) because it doesn't know where the destination is. cf line 73 of this example, where I'm using ping to check the lab status (and populating switch MAC table) on one of my script here: https://github.com/ocochard/netbenches/blob/master/Atom_C2758_8Cores-Chelsio_T540-CR/bench-lab-2nodes.config#L73 > > #####dut (DL360 G6) configuration: > #cxl0 MAC 00:07:43:2f:29:b0 > #cxl1 MAC 00:07:43:2f:29:b8 > ifconfig cxl0 198.18.0.203/24 > ifconfig cxl1 198.19.0.203/24 > #to create static arp records on restart use > sysrc static_arp_pairs="generator receiver" > sysrc static_arp_generator="198.18.0.201 00:07:43:32:b1:61" > sysrc static_arp_receiver="198.19.0.201 00:07:43:32:b1:69" > #to create static arp records on the fly (lost after restart) > arp -S 198.18.0.201 00:07:43:32:b1:61 > arp -S 198.19.0.201 00:07:43:32:b1:69 > #add a static route for 198.19.0.0/16 > route add -net 198.19.0.0/16 198.19.0.201 > > Did you disable ICMP redirect too on the router ? And the Chelsio advanced hardware feature ? Here is an example of my configuration file: https://github.com/ocochard/netbenches/tree/master/Atom_C2758_8Cores-Chelsio_T540-CR/forwarding-pf-ipfw/configs/forwarding > #####start the test > #start the receiver part > pkt-gen -N -f rx -i vcxl1 -w 4 > #start the generator part, use '-p 2' on order to achieve higher > throughput from the processor. > pkt-gen -N -f tx -i vcxl0 -n 1000000000 -4 -d 198.19.10.1:2000-198.19.10.100 > -D 00:07:43:2f:29:b0 -s 198.18.10.1:2000-198.18.10.20 -S > 00:07:43:32:b1:61 -w 4 -l 60 -U -p 2 > pkt-gen option -U (software checksum calculation) has a big performance impact (because my quick&dirty patch didn't use a pre-calculate table but calculate checksum for each packet generated) and Chelsio NIC (vcxl0 here) support doing hardware checksum calculation in netmap mode. > > > #####Results: > The generator can generate about 5.5 Mpps. > There is a problem here: With your hardware you should be able to generate line-rate (14.48Mpps). > The router drops about 1.5 Mpps. > The reciever gets about 4 Mpps. > Wow, this is not good because lower than a small Atom (8core) CPU: And this small Atom has a PDT of 20W comparing to this Xeon with a PDT of 80W. I never tested a Xeon E5410 with only 4 cores, but I would expected it be able to forward a lot more than 4Mpps. cf Atom results here: https://github.com/ocochard/netbenches/tree/master/Atom_C2758_8Cores-Chelsio_T540-CR/forwarding-pf-ipfw/results/fbsd12-stable.r354440.BSDRP.1.96 There is a 100% saturation on one of the CPU cores and about 75% evenly > distributed among the other cores on the router: > If the load is evenly distributed, this confirm you are correctly generating multi-flows. > > The generator can generate more than 10 Mpps but the number of flows > should be one or two for this hardware configuration: > pkt-gen -N -f tx -i vcxl0 -n 1000000000 -4 -d 198.19.10.1:2000-198.19.10.2 > -D 00:07:43:2f:29:b0 -s 198.18.10.1:2000-198.18.10.10 -S > 00:07:43:32:b1:61 -w 4 -l 60 -U -p 2 > > Try without -U, and it will be able to reach line-rate. Disable all unneeded Chelsio advanced feature too (check the loader.conf.local examples). Regards, Olivier
_______________________________________________ Bsdrp-users mailing list Bsdrp-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bsdrp-users
