Thomas has obtained the Ethernet captures, as requested in my previous post, and has emailed them to me for analysis with WireShark.
Thomas said he used the iperf option '-l 512' to force the size of the read/write buffer to be 512 (iperf default is normally 8192). So we have two capture files of iperf traffic, for 2 seconds duration, one with OpenSolaris as client, and the other with Linux as client. In both cases Linux is the server. Here is my preliminary assessment: :---------------------------------------: First file 'osol.iperf.cap' - OpenSolaris client. Approx 6MB file, capturing 62471 packets. The 1st packet, a SYN, shows MTU 1460, SACK permitted, Windows Scale Zero (*1) In WireShark, 'Analyze > Expert Info Composite' shows: Warning 'ACKed lost segment' count of 2079 Warning 'Previous segment lost' count of 398 Note 'Window update' count of 379 Note 'Window is full' count of 7 Data packets are of TCP size 512 (Ethernet 566), or 1024 (Ethernet 1078) So OpenSolaris is respecting the '-l 512' buffer size request. In WireShark, 'Statistics > TCP Stream Graph > Time-sequence graph' shows a curved line that starts diagonal, but then begins to level off. :---------------------------------------: Second file 'linux.iperf.cap' - Linux client. Approx 10MB file, capturing 104634 packets. The 1st packet, a SYN, shows MTU 1460, SACK permitted, Windows Scale 7 (*128), and Timestamps. Wireshark shows all packets are using the TCP timestamps option. In WireShark, 'Analyze > Expert Info Composite' shows: Warning 'ACKed lost segment' count of 7193 Warning 'Previous segment lost' count of 14786 There are no notes mentioning change in Window size. Data packets are all of TCP size 1448 (Ethernet 1514). So Linux does NOT seem to be respecting the '-l 512' request. In WireShark, 'Statistics > TCP Stream Graph > Time-sequence graph' shows a nice straight diagonal line. :---------------------------------------: The warnings about lost segments do not seem to be affecting performance. So the difference seem to be that Linux is using TCP timestamps, and the windows scaling option. And Linux performance is probably helped by not respecting the request for a 512 R/W buffer. Maybe we do need some help from experts across on the network-discuss mailing list to fully understand this. But I would say that it would be worth, on the OpenSolaris client, enabling tcp timestamps and window size scaling. I've asked Thomas if he would consent to making these capture files publicly available from my web site, so other can take a look and give an opinion. Thanks Nigel Smith -- This message posted from opensolaris.org _______________________________________________ storage-discuss mailing list storage-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/storage-discuss
