A follow-up. Please see below. Alex
> -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of > Serguei Osokine > Sent: Monday, December 17, 2007 9:52 AM > To: theory and practice of decentralized computer networks > Subject: Re: [p2p-hackers] MTU in the real world > > On Sunday, December 16, 2007 Alex Pankratov wrote: > > Admittedly, I didn't run the test over a one-gig link, but > > still the discrepancy between your findings and my results > > is a quite a bit odd. > > I'm not sure what's happening on 100 MBits/s. The real fun did > not even start until I was well over that number - the original > problem was to somehow send a gigabit per second of non-fragmented > non-jumbo UDP packets. I suspected just that. That's why I added the 100 Mbps disclaimer :) > [snip] > > > * the execution time of sendto() on my machine clearly depends on > > a size of the packet and it is virtually the same for blocking > > and non-blocking sockets. > > > > bytes microseconds > > 256 25 > > 1024 87 > > 4096 345 > > 16384 1370 > > In gigabit scenario you have to be under 10 microseconds per 1-KB > packet in order to fill the link to capacity. On our 1000-1400 byte > packets this time was several times higher than 10 microseconds, and > it was this time that was the performance bottleneck when a single > thread was doing all the sending. If multiple threads were sending > data, the CPU was maxing out and becoming the bottleneck instead. I run the same test over 1 Gbps, and I can now see constant execution time of the sendto. It's really bizarre. What's even more interesting, its behavior appears to change abruptly once the datagram size goes over 1024 bytes. sendto() for 1024 bytes or less runs at about 15 us/call sendto() for 1025 bytes and up - 244 us/call, constant And the problem goes away once the socket is unblocked. Just as you described. The call time stays at 15-17 us for the sizes up to 1470 bytes, so maxing the link capacity is still problematic. In fact, I tried this and that and I was not able to go over 69% utilization. The test app was still getting WOULDBLOCKs (in ~5% of calls), spawning multiple threads or processes did not make any difference, so I am inclined to think that this is a driver-level issue. It looks suspiciously like a driver that continues to use interrupts instead of polling under the high load. _______________________________________________ p2p-hackers mailing list [email protected] http://lists.zooko.com/mailman/listinfo/p2p-hackers
