Andrew, Bob: does Iperf use send() system call? Based on Iperf source code and tests with strace, it seems to use write() system call to send data to socket and according to write() manual page, it doesn't have a similar error to ENOBUFS. However, I could easily be wrong here.
Andrew, if Linux kernel silently drops packets in case the device queue overflows, then how does Iperf client know that? For example if I start the Iperf client with "iperf -c 10.10.10.1 -fm -t 600 -i 60 -u -b 500m" command and thus request Iperf to send data with bandwidth of 500Mbps and Iperf client reports me that it has sent data at 120Mbps instead of 500Mbps: [ 3] 0.0-600.0 sec 8602 MBytes 120 Mbits/sec ..then this means that Iperf client somehow had to know that some of the data was dropped or Iperf client was restricted to send data at faster rate than 120Mbps. How to explain this behavior? thanks, Martin On 9/8/14, Andrew Gallatin <galla...@gmail.com> wrote: > No, just that it is fairly new (added 5 years ago in "6ce9e7b ip: Report > qdisc packet drops"). Netperf did not have it either, until I tried to > figure out why I was getting wildly inaccurate results from netperf (eg, > seeing 20Gb/s on a 1GbE link). > > > On Mon, Sep 8, 2014 at 11:34 AM, Bob (Robert) McMahon > <rmcma...@broadcom.com >> wrote: > >> Hi Drew, >> >> >> >> Thanks for the detailed explanation. Do you know any reason that iperf >> doesn’t or shouldn’t set IP_RECVERR on the sending socket(s)? >> >> >> >> Thanks, >> >> Bob >> >> *From:* Andrew Gallatin [mailto:galla...@gmail.com] >> *Sent:* Monday, September 08, 2014 6:53 AM >> *To:* Martin T >> *Cc:* iperf-users >> *Subject:* Re: [Iperf-users] Iperf client sends out less UDP traffic than >> determined with "-b" flag >> >> >> >> Please, there are no "dropping" or "non dropping" NIC drivers, unless >> some >> driver is so broken that >> >> it silently returns NETDEV_TX_OK & does not stop its txqueue, but a >> driver >> that broken would >> >> not be accepted into the kernel. >> >> >> >> What is really happening is that the linux kernel is silently dropping >> packets when the device's >> >> queue is full *AND* when the device queue is too small to hold the entire >> socket buffer's worth >> >> of traffic. If you want blocking behavior, you need to ensure you have >> the driver's queue >> >> large enough (or the socket buffer small enough) so that the driver's >> queue can hold an >> >> entire socket buffer worth of traffic (* the number of active sockets per >> queue). You can change >> >> the driver's queue size via ifconfig txqueuelen. It might be better to >> reduce the socketbuffer >> >> size (and remember that linux multiplies whatever value you give it by 2, >> to account for >> >> slop...). >> >> >> >> Note that any silent drops are done by the kernel, not the NIC and that >> this is >> >> (bizarrely) the expected behavior on linux. If you don't believe me, >> read the send(2) man page on linux: >> >> ENOBUFS >> >> The output queue for a network interface was full. This >> generally indicates that the >> >> interface has stopped sending, but may be caused by >> transient congestion. (Normally, >> >> this does not occur in Linux. Packets are just silently >> dropped when a device queue >> >> overflows.) >> >> >> >> If iperf wants a correct accounting of the packets sent by the stack vs >> dropped, iperf must set >> >> the IP_RECVERR sockopt on the sending socket. This stops the kernel from >> silently >> >> dropping packets, and causes send() to return -ENOBUFS rather than 0 when >> >> datagram is dropped due to lack of resources (eg, like BSD does). >> >> >> >> Drew >> > ------------------------------------------------------------------------------ Want excitement? Manually upgrade your production database. When you want reliability, choose Perforce Perforce version control. Predictably reliable. http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk _______________________________________________ Iperf-users mailing list Iperf-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iperf-users
