Hi Martin, Yes, the value is either a positive integer (indicating the number of bytes written into kernel space) or a (negative value indicating some sort of socket write error.) The iperf client will terminate on all socket write errors except ENOBUF (which it silently ignores.) The write will preserve the UDP payload, i.e. the entire 1470 bytes will either be copied or not, depending upon kernel buffer space being available or not. No partial writes should occur.
As an FYI, just because the kernel has the UDP payload doesn't guarantee its successful transmission to the iperf server. If, for example, the ARP cache times out and cannot be updated the kernel will drop packets without informing the iperf client, and the iperf client won't be able to distinguish this from any other network outage. Also, in these cases, the client will include the packet in the transmit rate calculation because the kernel did accept the packet even though the ultimate transmit failed. Bob -----Original Message----- From: Martin T [mailto:[email protected]] Sent: Monday, September 22, 2014 3:32 PM To: Bob (Robert) McMahon Cc: Marc Herbert; [email protected] Subject: Re: [Iperf-users] Iperf client sends out less UDP traffic than determined with "-b" flag Bob, thanks for confirming this! In addition, am I correct that write return value can either be the number of bytes written or an error? In other words, following scenario is not possible where I write 1470 bytes, but return value is 970 bytes: [pid 14025] write(3,"\0\0\0\17T\n\331a\0\3\334\22\0\0\0\0\0\0\0\1\0\0\23\211\0\0\0\0\0\17B@"...,1470) = 970 Last but not least, are there any other possible reasons besides congested buffers for some write() system calls to fail? If yes, then what are those theoretical reasons? regards, Martin On Mon, Sep 22, 2014 at 8:04 AM, Bob (Robert) McMahon <[email protected]> wrote: > Hi Martin, > > Yes on the summing of the return values to get the transmit rate. > > Bob > -----Original Message----- > From: Martin T [mailto:[email protected]] > Sent: Sunday, September 21, 2014 1:31 PM > To: Marc Herbert; Bob (Robert) McMahon > Cc: [email protected] > Subject: Re: [Iperf-users] Iperf client sends out less UDP traffic than > determined with "-b" flag > > Marc, > >> In this case, what throughput number do you get on the server side? >> Different from the client or always the same? > > Throughput I see on a server side was always the same as Iperf client > reported, e.g. if Iperf client reported "[ 3] 0.0-600.0 sec 8602 > MBytes 120 Mbits/sec" then both Iperf server and switches between > the client and server saw 120Mbps of traffic. > > >> Is 120Mb/s the UDP maximum you ever see on the client, whatever parameters >> you use? > > Correct. > > >> What is the maximum UDP sending rate you ever saw reported on this specific >> hardware and operating system? > > It was a VMware virtual-machine(Debian Wheezy was a guest-OS) and > maximum UDP sending rate I received on this particular virtual-machine > was 120Mbps. If I replaced > the guest-OS in this very same virtual-machine with CentOS 7, the same > Iperf release with the same command was able to send traffic at > 500Mbps. > > > > > Bob, > > thank you for this very clear explanation! So for example in those two > write system calls: > > [pid 14025] > write(3,"\0\0\0\16T\n\331a\0\3\2563\0\0\0\0\0\0\0\1\0\0\23\211\0\0\0\0\0\17B@"...,1470) > = 1470 > [pid 14025] nanosleep({0, 11071000}, <unfinished ...> > [pid 14026] <... nanosleep resumed> NULL) = 0 > [pid 14026] nanosleep({0, 10000000}, NULL) = 0 > [pid 14026] nanosleep({0, 10000000}, <unfinished ...> > [pid 14025] <... nanosleep resumed> 0xb74ba2c8) = 0 > [pid 14025] gettimeofday({1409997153, 252946}, NULL) = 0 > [pid 14025] > write(3,"\0\0\0\17T\n\331a\0\3\334\22\0\0\0\0\0\0\0\1\0\0\23\211\0\0\0\0\0\17B@"...,1470) > = 1470 > [pid 14025] nanosleep({0, 11088000}, <unfinished ...> > [pid 14026] <... nanosleep resumed> NULL) = 0 > [pid 14026] nanosleep({0, 10000000}, <unfinished ...> > [pid 14025] <... nanosleep resumed> 0xb74ba2c8) = 0 > [pid 14025] gettimeofday({1409997153, 264721}, NULL) = 0 > > ..the number after the equals sign(1470) is the write return value aka > number of bytes accepted by kernel? And Iperf client just sums those > write() return values together and reports the amount of data > transmitted in bytes and in bps? > > > I also created a small program in C which writes a string into file > and here is the strace comparison when write() is successful: > > open("/media/file.txt", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3 > write(3, "Test string\n", 12) = 12 > exit_group(12) = ? > > ..and when the media is full: > > open("/media/file.txt", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3 > write(3, "Test string\n", 12) = -1 ENOSPC (No space left on device) > exit_group(-1) = ? > > > > regards, > Martin > > > On Fri, Sep 19, 2014 at 2:58 AM, Marc Herbert <[email protected]> wrote: >>> 2) Iperf client didn't send the data at faster rate than 120Mbps >>> because it received some error-signals from kernel and immediately >>> sent out datagrams less frequently >> >> Whether packets are lost inside the sender or not and no matter how hard it >> tries, the client cannot perform an infinite number of write() per seconds. >> There is a "physical" limit caused by either the CPU, or memory bandwidth, >> or scheduling, etc. What is the maximum UDP sending rate you ever saw >> reported on this specific hardware and operating system? >> >> As already suggested on this list a few times over the years, also try >> varying the socket buffer size and see if it makes any difference. >> >> Marc >> >> 2014-09-18 11:52 GMT+01:00 Martin T <[email protected]>: >>> >>> Marc, >>> >>> I'm afraid I didn't understand your question.. My question has nothing >>> to do with packet loss on the network. In case of packet loss on the >>> network, you see this in server-report if we speak about UDP mode. >>> >>> >>> I'll try to explain once again what I'm confused about. If I start the >>> Iperf client with "iperf -c 10.10.10.1 -fm -t 600 -i 60 -u -b 500m" >>> command and thus request Iperf to send data with bandwidth of 500Mbps >>> and Iperf client reports me that it has sent data at 120Mbps instead >>> of 500Mbps: >>> >>> [ 3] 0.0-600.0 sec 8602 MBytes 120 Mbits/sec >>> >>> ..then this means that Iperf client somehow had to know that some of >>> the data was either: >>> >>> 1) dropped in the local machine, i.e. Iperf client writes to socket at >>> 500Mbps, but before printing the "[ 3] 0.0-600.0 sec 8602 MBytes >>> 120 Mbits/sec" line, it somehow checks that actually not all of the >>> data written to socket was not transmitted >>> >>> 2) Iperf client didn't send the data at faster rate than 120Mbps >>> because it received some error-signals from kernel and immediately >>> sent out datagrams less frequently >>> >>> There is nothing to do with packet loss on network. >>> >>> >>> >>> regards, >>> Martin >>> >>> >>> On 9/18/14, Marc Herbert <[email protected]> wrote: >>> > Martin, >>> > >>> > What makes you think iperf counts local packet losses differently from >>> > network packet losses? >>> > >>> > Cheers, >>> > >>> > Marc >>> > >>> > 2014-09-18 8:50 GMT+01:00 Martin T <[email protected]>: >>> > >>> >> Andrew, >>> >> >>> >> I'll rephrase my question which I asked earlier. If Iperf ignores the >>> >> errors regarding lack of memory in interface buffers, then how does >>> >> the Iperf client know that under those conditions it should make the >>> >> write calls to socket less frequently? For example if I start the >>> >> Iperf client with "iperf -c 10.10.10.1 -fm -t 600 -i 60 -u -b 500m" >>> >> command and thus request Iperf to send data with bandwidth of 500Mbps >>> >> and Iperf client reports me that it has sent data at 120Mbps instead >>> >> of 500Mbps: >>> >> >>> >> [ 3] 0.0-600.0 sec 8602 MBytes 120 Mbits/sec >>> >> >>> >> ..then this means that Iperf client somehow had to know that some of >>> >> the data was dropped or Iperf client didn't send the data at faster >>> >> rate than 120Mbps. >>> >> >>> >> >>> >> thanks, >>> >> Martin >>> >> >>> >> On 9/17/14, Andrew Gallatin <[email protected]> wrote: >>> >> > Actually, there are probably ways you could tell. But the simplest >>> >> > is >>> >> > to >>> >> > just keep re-sending until you don't get an error, which is what >>> >> benchmarks >>> >> > do. >>> >> > >>> >> > On Wed, Sep 17, 2014 at 7:53 AM, Andrew Gallatin <[email protected]> >>> >> > wrote: >>> >> > >>> >> >> There is no way to know. UDP is lossy, after all. >>> >> >> >>> >> >> On Wed, Sep 17, 2014 at 7:47 AM, Martin T <[email protected]> >>> >> >> wrote: >>> >> >> >>> >> >>> Andrew, >>> >> >>> >>> >> >>> if Iperf ignores the errors regarding lack of memory in interface >>> >> >>> buffers, then how does Iperf know that it should send out the >>> >> >>> datagrams less frequently, i.e. write to socket less frequently? >>> >> >>> >>> >> >>> >>> >> >>> thanks, >>> >> >>> Martin >>> >> >>> >>> >> >>> >>> >> >>> On 9/17/14, Andrew Gallatin <[email protected]> wrote: >>> >> >>> > In my (cursory) read of the 3.x code, it looked like it >>> >> >>> > ignores NET_SOFTERROR returns & retries. It would be nice if >>> >> >>> > it >>> >> >>> reported >>> >> >>> > the amount of ENOBUF errors it received like netperf does.. >>> >> >>> > >>> >> >>> > On Wed, Sep 17, 2014 at 4:09 AM, Martin T <[email protected]> >>> >> wrote: >>> >> >>> > >>> >> >>> >> Andrew, Bob, >>> >> >>> >> >>> >> >>> >> did I understand you correctly that while ENOBUFS error code is >>> >> >>> >> not >>> >> >>> >> shown in Iperf client output, the Iperf client actually takes it >>> >> into >>> >> >>> >> account and thus sends data out less frequently? >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> regards, >>> >> >>> >> Martin >>> >> >>> >> >>> >> >>> >> On 9/11/14, Bob (Robert) McMahon <[email protected]> wrote: >>> >> >>> >> > I thought I remembered hitting the ENOBUFs which is the case >>> >> >>> >> > per >>> >> >>> >> > the >>> >> >>> >> > source. >>> >> >>> >> > >>> >> >>> >> > // perform write >>> >> >>> >> > currLen = write( mSettings->mSock, mBuf, mSettings->mBufLen ); >>> >> >>> >> > if ( currLen < 0 && errno != ENOBUFS ) { >>> >> >>> >> > WARN_errno( currLen < 0, "write2" ); >>> >> >>> >> > break; >>> >> >>> >> > } >>> >> >>> >> > // report packets >>> >> >>> >> > reportstruct->packetLen = currLen; >>> >> >>> >> > ReportPacket( mSettings->reporthdr, reportstruct ); >>> >> >>> >> > >>> >> >>> >> > A minor nit, the currLen may need to set to zero when >>> >> >>> >> > errno==ENOBUFFS >>> >> >>> >> before >>> >> >>> >> > the ReportPacket() >>> >> >>> >> > >>> >> >>> >> > Also, ione could count the number of writes that return >>> >> >>> >> > ENOBUFFS >>> >> >>> >> > but >>> >> >>> >> > not >>> >> >>> >> > sure why that's interesting. >>> >> >>> >> > >>> >> >>> >> > As an FYI, I have a tool that manages multiple, distributed >>> >> >>> >> > iperf >>> >> >>> >> sessions >>> >> >>> >> > which I use for wi-fi testing. The following shows an 802.11 >>> >> >>> >> > AC >>> >> >>> wi-fi >>> >> >>> >> link >>> >> >>> >> > that maxes out around 800Mbs. The offered load is 2G and the >>> >> iperf >>> >> >>> >> client >>> >> >>> >> > seems to be reporting properly and responding as expected when >>> >> >>> >> > RF >>> >> >>> >> > attenuation is added and then removed. >>> >> >>> >> > >>> >> >>> >> > utf> udp start; udp linkcheck -now; L1 attn 65; UTF::Sleep >>> >> >>> >> > 0.3; >>> >> L1 >>> >> >>> >> > attn >>> >> >>> >> 0; >>> >> >>> >> > UTF::Sleep 0.3; udp stop >>> >> >>> >> > 20:45:08 LOG sfast-lx2 iperf -s -w 7350000 -l 1470 -u -i >>> >> >>> >> > 0.1 >>> >> >>> >> > -fb >>> >> >>> >> > -p >>> >> >>> >> > 60001 >>> >> >>> >> > 20:45:08 HNDLR_ udp-rx Server listening on UDP port 60001 >>> >> >>> >> > (sflx2,file12) >>> >> >>> >> > 20:45:08 HNDLR_ udp-rx Receiving 1470 byte datagrams >>> >> >>> >> > (sflx2,file12) >>> >> >>> >> > 20:45:08 HNDLR_ udp-rx UDP buffer size: 8388608 Byte >>> >> (WARNING: >>> >> >>> >> > requested 7350000 Byte) (sflx2,file12) >>> >> >>> >> > 20:45:08 LOG itx-lx1 iperf -c 192.168.1.42 -w 367500 -l >>> >> >>> >> > 1470 >>> >> >>> >> > -u >>> >> >>> >> > -b >>> >> >>> >> 2G >>> >> >>> >> > -B 192.168.1.100 -i 0.1 -fb -S 0x0 -T 255 -t 5356800 -p 60001 >>> >> >>> >> > 20:45:08 HNDLR udp-tx >>> >> >>> >> > ------------------------------------------------------------ >>> >> >>> >> > (43602lx1,file13) >>> >> >>> >> > 20:45:08 HNDLR_ udp-rx [ 3] local 192.168.1.42 port >>> >> >>> >> > 60001 >>> >> >>> >> > connected >>> >> >>> >> > with 192.168.1.100 port 60001 (sflx2,file12) >>> >> >>> >> > 20:45:08 HNDLR udp-tx Client connecting to 192.168.1.42, >>> >> >>> >> > UDP >>> >> >>> port >>> >> >>> >> 60001 >>> >> >>> >> > (43602lx1,file13) >>> >> >>> >> > 20:45:08 INFO udp-tx TX start: protocol=UDP Offered >>> >> >>> >> > rate=2G >>> >> >>> >> > pktsize=1470 delay=5us >>> >> >>> >> > 20:45:08 HNDLR udp-tx Binding to local address >>> >> >>> >> > 192.168.1.100 >>> >> >>> >> > (43602lx1,file13) >>> >> >>> >> > 20:45:08 HNDLR udp-tx Sending 1470 byte datagrams >>> >> >>> >> > (43602lx1,file13) >>> >> >>> >> > 20:45:08 HNDLR udp-tx UDP buffer size: 735000 Byte >>> >> >>> >> > (WARNING: >>> >> >>> >> requested >>> >> >>> >> > 367500 Byte) (43602lx1,file13) >>> >> >>> >> > 20:45:08 HNDLR udp-tx >>> >> >>> >> > ------------------------------------------------------------ >>> >> >>> >> > (43602lx1,file13) >>> >> >>> >> > 20:45:08 HNDLR udp-tx [ 3] local 192.168.1.100 port >>> >> >>> >> > 60001 >>> >> >>> >> > connected >>> >> >>> >> > with 192.168.1.42 port 60001 (43602lx1,file13) >>> >> >>> >> > 20:45:08 HNDLR udp-tx [ ID] Interval Transfer >>> >> >>> Bandwidth >>> >> >>> >> > (43602lx1,file13) >>> >> >>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.000-0.100 sec 11244030 >>> >> >>> >> > Bytes >>> >> >>> >> 899522400 >>> >> >>> >> > bits/sec (43602lx1,file13) >>> >> >>> >> > 20:45:08 LOG L1 attn 65 >>> >> >>> >> > 20:45:08 INFO ::AF-A Setting attn to 65 >>> >> >>> >> > 20:45:08 INFO Sleep 0.3 sec >>> >> >>> >> > 20:45:08 LOG itx-lx1 wl0.0: wlc_send_bar: seq 0x17f tid >>> >> >>> >> > 0 >>> >> >>> >> > 20:45:08 LOG itx-lx1 wl0.0: wlc_send_bar: seq 0x17f tid >>> >> >>> >> > 0 >>> >> >>> >> (repeated 1 >>> >> >>> >> > times) >>> >> >>> >> > 20:45:08 LOG L1 attn 0 >>> >> >>> >> > 20:45:08 INFO ::AF-A Setting attn to 0 >>> >> >>> >> > 20:45:08 INFO Sleep 0.3 sec >>> >> >>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.100-0.200 sec 2625420 >>> >> >>> >> > Bytes >>> >> >>> >> 210033600 >>> >> >>> >> > bits/sec (43602lx1,file13) >>> >> >>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.200-0.300 sec 0.00 Bytes >>> >> >>> >> > 0.00 >>> >> >>> >> bits/sec >>> >> >>> >> > (43602lx1,file13) >>> >> >>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.300-0.400 sec 0.00 Bytes >>> >> >>> >> > 0.00 >>> >> >>> >> bits/sec >>> >> >>> >> > (43602lx1,file13) >>> >> >>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.400-0.500 sec 6251910 >>> >> >>> >> > Bytes >>> >> >>> >> 500152800 >>> >> >>> >> > bits/sec (43602lx1,file13) >>> >> >>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.500-0.600 sec 10406130 >>> >> >>> >> > Bytes >>> >> >>> >> 832490400 >>> >> >>> >> > bits/sec (43602lx1,file13) >>> >> >>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.600-0.700 sec 10229730 >>> >> >>> >> > Bytes >>> >> >>> >> 818378400 >>> >> >>> >> > bits/sec (43602lx1,file13) >>> >> >>> >> > 20:45:08 INFO udp Kill signal -HUP sent to >>> >> >>> (43602lx1,file13) : >>> >> >>> >> > iperf -c 192.168.1.42 -w 367500 -l 1470 -u -b 2G -B >>> >> >>> >> > 192.168.1.100 >>> >> >>> >> > -i >>> >> >>> >> > 0.1 >>> >> >>> >> > -fb -S 0x0 -T 255 -t 5356800 -p 60001 >>> >> >>> >> > 20:45:09 INFO udp Kill signal -HUP sent to >>> >> >>> >> > (sflx2,file12) >>> >> >>> >> > : >>> >> >>> >> iperf >>> >> >>> >> > -s -w 7350000 -l 1470 -u -i 0.1 -fb -p 60001 >>> >> >>> >> > 20:45:09 HNDLR udp-tx Close actions for event handler >>> >> >>> >> > (43602lx1,file13) >>> >> >>> >> > 20:45:09 INFO udp-tx Closed : (43602lx1,file13) >>> >> >>> >> > >>> >> >>> >> > Bob >>> >> >>> >> > -----Original Message----- >>> >> >>> >> > From: Bob (Robert) McMahon >>> >> >>> >> > Sent: Wednesday, September 10, 2014 5:02 PM >>> >> >>> >> > To: Martin T; [email protected] >>> >> >>> >> > Cc: [email protected] >>> >> >>> >> > Subject: RE: [Iperf-users] Iperf client sends out less UDP >>> >> >>> >> > traffic >>> >> >>> than >>> >> >>> >> > determined with "-b" flag >>> >> >>> >> > >>> >> >>> >> > Hi Martin, >>> >> >>> >> > >>> >> >>> >> > Just to confirm Andrew, I'm pretty sure I've seen the write >>> >> syscall >>> >> >>> on >>> >> >>> >> linux >>> >> >>> >> > return ENOBUFFs. I'll check it out in the next few days and >>> >> >>> >> > let >>> >> >>> >> > you >>> >> >>> >> know. >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> >> > Bob >>> >> >>> >> > -----Original Message----- >>> >> >>> >> > From: Martin T [mailto:[email protected]] >>> >> >>> >> > Sent: Wednesday, September 10, 2014 5:12 AM >>> >> >>> >> > To: [email protected]; Bob (Robert) McMahon >>> >> >>> >> > Cc: [email protected] >>> >> >>> >> > Subject: Re: [Iperf-users] Iperf client sends out less UDP >>> >> >>> >> > traffic >>> >> >>> than >>> >> >>> >> > determined with "-b" flag >>> >> >>> >> > >>> >> >>> >> > Andrew, Bob: >>> >> >>> >> > >>> >> >>> >> > does Iperf use send() system call? Based on Iperf source code >>> >> >>> >> > and >>> >> >>> >> > tests with strace, it seems to use write() system call to send >>> >> data >>> >> >>> to >>> >> >>> >> > socket and according to write() manual page, it doesn't have a >>> >> >>> similar >>> >> >>> >> > error to ENOBUFS. However, I could easily be wrong here. >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> >> > Andrew, >>> >> >>> >> > >>> >> >>> >> > if Linux kernel silently drops packets in case the device >>> >> >>> >> > queue >>> >> >>> >> > overflows, then how does Iperf client know that? For example >>> >> >>> >> > if >>> >> >>> >> > I >>> >> >>> >> > start the Iperf client with "iperf -c 10.10.10.1 -fm -t 600 -i >>> >> >>> >> > 60 >>> >> >>> >> > -u >>> >> >>> >> > -b 500m" command and thus request Iperf to send data with >>> >> bandwidth >>> >> >>> of >>> >> >>> >> > 500Mbps and Iperf client reports me that it has sent data at >>> >> >>> >> > 120Mbps >>> >> >>> >> > instead of 500Mbps: >>> >> >>> >> > >>> >> >>> >> > [ 3] 0.0-600.0 sec 8602 MBytes 120 Mbits/sec >>> >> >>> >> > >>> >> >>> >> > ..then this means that Iperf client somehow had to know that >>> >> >>> >> > some >>> >> >>> >> > of >>> >> >>> >> > the data was dropped or Iperf client was restricted to send >>> >> >>> >> > data >>> >> at >>> >> >>> >> > faster rate than 120Mbps. How to explain this behavior? >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> >> > thanks, >>> >> >>> >> > Martin >>> >> >>> >> > >>> >> >>> >> > On 9/8/14, Andrew Gallatin <[email protected]> wrote: >>> >> >>> >> >> No, just that it is fairly new (added 5 years ago in "6ce9e7b >>> >> >>> >> >> ip: >>> >> >>> >> >> Report >>> >> >>> >> >> qdisc packet drops"). Netperf did not have it either, until >>> >> >>> >> >> I >>> >> >>> >> >> tried >>> >> >>> >> >> to >>> >> >>> >> >> figure out why I was getting wildly inaccurate results from >>> >> >>> >> >> netperf >>> >> >>> >> >> (eg, >>> >> >>> >> >> seeing 20Gb/s on a 1GbE link). >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> On Mon, Sep 8, 2014 at 11:34 AM, Bob (Robert) McMahon >>> >> >>> >> >> <[email protected] >>> >> >>> >> >>> wrote: >>> >> >>> >> >> >>> >> >>> >> >>> Hi Drew, >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> Thanks for the detailed explanation. Do you know any reason >>> >> that >>> >> >>> >> >>> iperf >>> >> >>> >> >>> doesn’t or shouldn’t set IP_RECVERR on the sending >>> >> >>> >> >>> socket(s)? >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> Thanks, >>> >> >>> >> >>> >>> >> >>> >> >>> Bob >>> >> >>> >> >>> >>> >> >>> >> >>> *From:* Andrew Gallatin [mailto:[email protected]] >>> >> >>> >> >>> *Sent:* Monday, September 08, 2014 6:53 AM >>> >> >>> >> >>> *To:* Martin T >>> >> >>> >> >>> *Cc:* iperf-users >>> >> >>> >> >>> *Subject:* Re: [Iperf-users] Iperf client sends out less UDP >>> >> >>> traffic >>> >> >>> >> >>> than >>> >> >>> >> >>> determined with "-b" flag >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> Please, there are no "dropping" or "non dropping" NIC >>> >> >>> >> >>> drivers, >>> >> >>> unless >>> >> >>> >> >>> some >>> >> >>> >> >>> driver is so broken that >>> >> >>> >> >>> >>> >> >>> >> >>> it silently returns NETDEV_TX_OK & does not stop its >>> >> >>> >> >>> txqueue, >>> >> but >>> >> >>> >> >>> a >>> >> >>> >> >>> driver >>> >> >>> >> >>> that broken would >>> >> >>> >> >>> >>> >> >>> >> >>> not be accepted into the kernel. >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> What is really happening is that the linux kernel is >>> >> >>> >> >>> silently >>> >> >>> >> >>> dropping >>> >> >>> >> >>> packets when the device's >>> >> >>> >> >>> >>> >> >>> >> >>> queue is full *AND* when the device queue is too small to >>> >> >>> >> >>> hold >>> >> >>> >> >>> the >>> >> >>> >> >>> entire >>> >> >>> >> >>> socket buffer's worth >>> >> >>> >> >>> >>> >> >>> >> >>> of traffic. If you want blocking behavior, you need to >>> >> >>> >> >>> ensure >>> >> >>> >> >>> you >>> >> >>> >> >>> have >>> >> >>> >> >>> the driver's queue >>> >> >>> >> >>> >>> >> >>> >> >>> large enough (or the socket buffer small enough) so that the >>> >> >>> driver's >>> >> >>> >> >>> queue can hold an >>> >> >>> >> >>> >>> >> >>> >> >>> entire socket buffer worth of traffic (* the number of >>> >> >>> >> >>> active >>> >> >>> sockets >>> >> >>> >> >>> per >>> >> >>> >> >>> queue). You can change >>> >> >>> >> >>> >>> >> >>> >> >>> the driver's queue size via ifconfig txqueuelen. It might >>> >> >>> >> >>> be >>> >> >>> better >>> >> >>> >> >>> to >>> >> >>> >> >>> reduce the socketbuffer >>> >> >>> >> >>> >>> >> >>> >> >>> size (and remember that linux multiplies whatever value you >>> >> >>> >> >>> give >>> >> >>> >> >>> it >>> >> >>> >> >>> by >>> >> >>> >> >>> 2, >>> >> >>> >> >>> to account for >>> >> >>> >> >>> >>> >> >>> >> >>> slop...). >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> Note that any silent drops are done by the kernel, not the >>> >> >>> >> >>> NIC >>> >> >>> >> >>> and >>> >> >>> >> >>> that >>> >> >>> >> >>> this is >>> >> >>> >> >>> >>> >> >>> >> >>> (bizarrely) the expected behavior on linux. If you don't >>> >> believe >>> >> >>> me, >>> >> >>> >> >>> read the send(2) man page on linux: >>> >> >>> >> >>> >>> >> >>> >> >>> ENOBUFS >>> >> >>> >> >>> >>> >> >>> >> >>> The output queue for a network interface was >>> >> >>> >> >>> full. >>> >> >>> >> >>> This >>> >> >>> >> >>> generally indicates that the >>> >> >>> >> >>> >>> >> >>> >> >>> interface has stopped sending, but may be >>> >> >>> >> >>> caused >>> >> by >>> >> >>> >> >>> transient congestion. (Normally, >>> >> >>> >> >>> >>> >> >>> >> >>> this does not occur in Linux. Packets are >>> >> >>> >> >>> just >>> >> >>> >> >>> silently >>> >> >>> >> >>> dropped when a device queue >>> >> >>> >> >>> >>> >> >>> >> >>> overflows.) >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> If iperf wants a correct accounting of the packets sent by >>> >> >>> >> >>> the >>> >> >>> stack >>> >> >>> >> >>> vs >>> >> >>> >> >>> dropped, iperf must set >>> >> >>> >> >>> >>> >> >>> >> >>> the IP_RECVERR sockopt on the sending socket. This stops >>> >> >>> >> >>> the >>> >> >>> kernel >>> >> >>> >> >>> from >>> >> >>> >> >>> silently >>> >> >>> >> >>> >>> >> >>> >> >>> dropping packets, and causes send() to return -ENOBUFS >>> >> >>> >> >>> rather >>> >> >>> >> >>> than >>> >> >>> 0 >>> >> >>> >> >>> when >>> >> >>> >> >>> >>> >> >>> >> >>> datagram is dropped due to lack of resources (eg, like BSD >>> >> does). >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> >>> >> >>> >> >>> Drew >>> >> >>> >> >>> >>> >> >>> >> >> >>> >> >>> >> > >>> >> >>> >> >>> >> >>> > >>> >> >>> >>> >> >> >>> >> >> >>> >> > >>> >> >>> >> >>> >> >>> >> ------------------------------------------------------------------------------ >>> >> Want excitement? >>> >> Manually upgrade your production database. >>> >> When you want reliability, choose Perforce >>> >> Perforce version control. Predictably reliable. >>> >> >>> >> >>> >> http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk >>> >> _______________________________________________ >>> >> Iperf-users mailing list >>> >> [email protected] >>> >> https://lists.sourceforge.net/lists/listinfo/iperf-users >>> >> >>> > >> >> ------------------------------------------------------------------------------ Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk _______________________________________________ Iperf-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/iperf-users
