Re: [ofa-general] ipoib performance (and xplot)
you capture the transaction with tcpdump and use a perl script provided with xplot to convert it into something xplot can graph. megabits of course. so yeah, its pretty awful. and indeed the recv end of the tcp connection has saturated the cpu at 100%. apparently, this issue is further complicated by my architecture, the ia64. there doesnt seem to be an accelerated checksum/copy_to_user. so the ia64 makes two passes through the data. i should have suspected this. so the xplot graph was right. the ack's are coming back slow. thanks! In message <[EMAIL PROTECTED]>,"Talpey, Thomas " writes: >Interesting data. What app are you using to generate the TCP flow, >and what options are you using on it? Also, what are the scales on >the x- and y-axes (seconds and decimal kilobytes)? I have some >comments but they are only speculation without knowing this. > >By "800Mb/s" do you megabytes or megabits? For ipoib, 800 megabytes/s >(MB/s) seems very high and 800 megabits/s (Mb/s) seems very low. In my >experience it gets 200-300 megabytes before running out of cpu (checksum >calculations mainly). But I haven't looked at it in a while. > >Tom. ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] ipoib performance (and xplot)
Interesting data. What app are you using to generate the TCP flow, and what options are you using on it? Also, what are the scales on the x- and y-axes (seconds and decimal kilobytes)? I have some comments but they are only speculation without knowing this. By "800Mb/s" do you megabytes or megabits? For ipoib, 800 megabytes/s (MB/s) seems very high and 800 megabits/s (Mb/s) seems very low. In my experience it gets 200-300 megabytes before running out of cpu (checksum calculations mainly). But I haven't looked at it in a while. Tom. At 03:07 PM 3/6/2007, chas williams - CONTRACTOR wrote: >while looking at some ipoib performance i had a chance to graph the >tcp flow in xplot (see http://www.xplot.org/). the graph appears very >strange and is attached to this message. > >the lower solid line represent acks coming back from the tcp server, the >up line represent the window size (i disabled window scaling btw, this >doesnt affect performance). the up/down arrows (they look like diamonds >due to scale) represent packets. this is a view from the tcp client. > >the initial part of the graph is tcp slow-start/congestion. what is >curious to how the returning acks (after slow start is finished) seem >to get quiet periodically. then the next ack that returns, then acks >the entire window. this seems to be leading to a very bursty behavior. > >i would normally expect to see two data packets followed by an >ack as can be seen between the 'burst' regions. > >i see aboue 800Mb/s (good put) between the hosts which i understand >to get typical for ipoib. there dont appear to be any link errors >either. > >so why the long pauses? > >ftp://ftp.cmf.nrl.navy.mil/pub/chas/ipoib.jpg >___ >general mailing list >[email protected] >http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
