Hi Chris. How much improvement do you get with jumbo frames?
Can you achieve significantly higher output if you try multiple streams in parallel? During the test are there any CPU cores with very low idle time? Depending on the answers to the above it might be interesting to see if there is any improvement by increasing rx_queue_number and tx_queue_number on the ixgbe driver. Regards, Nick On 19 July 2014 14:42, Chris Ferebee via smartos-discuss < [email protected]> wrote: > > I'm trying to debug a network performance issue. > > I have two servers running SmartOS (20140613T024634Z and > 20140501T225642Z), one is a Supermicro dual Xeon E5649 (64 GB RAM) and the > other is a dual Xeon E5-2620v2 (128 GB RAM). Each has an Intel X520-DA1 > 10GbE card, and they are both connected to 10GbE ports on a NetGear > GS752TXS switch. > > The switch reports 10GbE links: > > 1/xg49 Enable 10G Full 10G Full Link Up > Enable 1518 20:0C:C8:46:C8:3E 49 49 > 1/xg50 Enable 10G Full 10G Full Link Up > Enable 1518 20:0C:C8:46:C8:3E 50 50 > > as do both hosts: > > [root@90-e2-ba-00-2a-e2 ~]# dladm show-phys > LINK MEDIA STATE SPEED DUPLEX DEVICE > igb0 Ethernet down 0 > half igb0 > igb1 Ethernet down 0 > half igb1 > ixgbe0 Ethernet up 10000 full > ixgbe0 > > [root@00-1b-21-bf-e1-b4 ~]# dladm show-phys > LINK MEDIA STATE SPEED DUPLEX DEVICE > igb0 Ethernet down 0 > half igb0 > ixgbe0 Ethernet up 10000 full > ixgbe0 > igb1 Ethernet down 0 > half igb1 > > Per dladm show-linkprop, maxbw is not set on either of the net0 vnic > interfaces. > > And yet, as measured via netcat, throughput is just below 1 Gbit/s: > > [root@90-e2-ba-00-2a-e2 ~]# time cat /zones/test/10gb | nc -v -v -n > 192.168.168.5 8888 > Connection to 192.168.168.5 8888 port [tcp/*] succeeded! > > real 1m34.662s > user 0m11.422s > sys 1m53.957s > > (In this test, 10gb is a test file that is warm in RAM and transfers via > dd to /dev/null at approx. 2.4 GByte/s.) > > What could be causing the slowdown, and how might I go about debugging > this? > > FTR, disk throughput, while not an issue here, appears to be perfectly > reasonable, approx. 900 MB/s read performance. > > Thanks for any pointers! > > Chris > > > > > ------------------------------------------- > smartos-discuss > Archives: https://www.listbox.com/member/archive/184463/=now > RSS Feed: > https://www.listbox.com/member/archive/rss/184463/22416839-083bd2e9 > Modify Your Subscription: > https://www.listbox.com/member/?&; > Powered by Listbox: http://www.listbox.com > ------------------------------------------- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00 Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb Powered by Listbox: http://www.listbox.com
