Andrew,
I'll rephrase my question which I asked earlier. If Iperf ignores the
errors regarding lack of memory in interface buffers, then how does
the Iperf client know that under those conditions it should make the
write calls to socket less frequently? For example if I start the
Iperf client with "iperf -c 10.10.10.1 -fm -t 600 -i 60 -u -b 500m"
command and thus request Iperf to send data with bandwidth of 500Mbps
and Iperf client reports me that it has sent data at 120Mbps instead
of 500Mbps:
[ 3] 0.0-600.0 sec 8602 MBytes 120 Mbits/sec
..then this means that Iperf client somehow had to know that some of
the data was dropped or Iperf client didn't send the data at faster
rate than 120Mbps.
thanks,
Martin
On 9/17/14, Andrew Gallatin <galla...@gmail.com> wrote:
> Actually, there are probably ways you could tell. But the simplest is to
> just keep re-sending until you don't get an error, which is what benchmarks
> do.
>
> On Wed, Sep 17, 2014 at 7:53 AM, Andrew Gallatin <galla...@gmail.com>
> wrote:
>
>> There is no way to know. UDP is lossy, after all.
>>
>> On Wed, Sep 17, 2014 at 7:47 AM, Martin T <m4rtn...@gmail.com> wrote:
>>
>>> Andrew,
>>>
>>> if Iperf ignores the errors regarding lack of memory in interface
>>> buffers, then how does Iperf know that it should send out the
>>> datagrams less frequently, i.e. write to socket less frequently?
>>>
>>>
>>> thanks,
>>> Martin
>>>
>>>
>>> On 9/17/14, Andrew Gallatin <galla...@gmail.com> wrote:
>>> > In my (cursory) read of the 3.x code, it looked like it
>>> > ignores NET_SOFTERROR returns & retries. It would be nice if it
>>> reported
>>> > the amount of ENOBUF errors it received like netperf does..
>>> >
>>> > On Wed, Sep 17, 2014 at 4:09 AM, Martin T <m4rtn...@gmail.com> wrote:
>>> >
>>> >> Andrew, Bob,
>>> >>
>>> >> did I understand you correctly that while ENOBUFS error code is not
>>> >> shown in Iperf client output, the Iperf client actually takes it into
>>> >> account and thus sends data out less frequently?
>>> >>
>>> >>
>>> >> regards,
>>> >> Martin
>>> >>
>>> >> On 9/11/14, Bob (Robert) McMahon <rmcma...@broadcom.com> wrote:
>>> >> > I thought I remembered hitting the ENOBUFs which is the case per
>>> >> > the
>>> >> > source.
>>> >> >
>>> >> > // perform write
>>> >> > currLen = write( mSettings->mSock, mBuf, mSettings->mBufLen );
>>> >> > if ( currLen < 0 && errno != ENOBUFS ) {
>>> >> > WARN_errno( currLen < 0, "write2" );
>>> >> > break;
>>> >> > }
>>> >> > // report packets
>>> >> > reportstruct->packetLen = currLen;
>>> >> > ReportPacket( mSettings->reporthdr, reportstruct );
>>> >> >
>>> >> > A minor nit, the currLen may need to set to zero when
>>> >> > errno==ENOBUFFS
>>> >> before
>>> >> > the ReportPacket()
>>> >> >
>>> >> > Also, ione could count the number of writes that return ENOBUFFS
>>> >> > but
>>> >> > not
>>> >> > sure why that's interesting.
>>> >> >
>>> >> > As an FYI, I have a tool that manages multiple, distributed iperf
>>> >> sessions
>>> >> > which I use for wi-fi testing. The following shows an 802.11 AC
>>> wi-fi
>>> >> link
>>> >> > that maxes out around 800Mbs. The offered load is 2G and the iperf
>>> >> client
>>> >> > seems to be reporting properly and responding as expected when RF
>>> >> > attenuation is added and then removed.
>>> >> >
>>> >> > utf> udp start; udp linkcheck -now; L1 attn 65; UTF::Sleep 0.3; L1
>>> >> > attn
>>> >> 0;
>>> >> > UTF::Sleep 0.3; udp stop
>>> >> > 20:45:08 LOG sfast-lx2 iperf -s -w 7350000 -l 1470 -u -i 0.1
>>> >> > -fb
>>> >> > -p
>>> >> > 60001
>>> >> > 20:45:08 HNDLR_ udp-rx Server listening on UDP port 60001
>>> >> > (sflx2,file12)
>>> >> > 20:45:08 HNDLR_ udp-rx Receiving 1470 byte datagrams
>>> >> > (sflx2,file12)
>>> >> > 20:45:08 HNDLR_ udp-rx UDP buffer size: 8388608 Byte (WARNING:
>>> >> > requested 7350000 Byte) (sflx2,file12)
>>> >> > 20:45:08 LOG itx-lx1 iperf -c 192.168.1.42 -w 367500 -l 1470
>>> >> > -u
>>> >> > -b
>>> >> 2G
>>> >> > -B 192.168.1.100 -i 0.1 -fb -S 0x0 -T 255 -t 5356800 -p 60001
>>> >> > 20:45:08 HNDLR udp-tx
>>> >> > ------------------------------------------------------------
>>> >> > (43602lx1,file13)
>>> >> > 20:45:08 HNDLR_ udp-rx [ 3] local 192.168.1.42 port 60001
>>> >> > connected
>>> >> > with 192.168.1.100 port 60001 (sflx2,file12)
>>> >> > 20:45:08 HNDLR udp-tx Client connecting to 192.168.1.42, UDP
>>> port
>>> >> 60001
>>> >> > (43602lx1,file13)
>>> >> > 20:45:08 INFO udp-tx TX start: protocol=UDP Offered rate=2G
>>> >> > pktsize=1470 delay=5us
>>> >> > 20:45:08 HNDLR udp-tx Binding to local address 192.168.1.100
>>> >> > (43602lx1,file13)
>>> >> > 20:45:08 HNDLR udp-tx Sending 1470 byte datagrams
>>> >> > (43602lx1,file13)
>>> >> > 20:45:08 HNDLR udp-tx UDP buffer size: 735000 Byte (WARNING:
>>> >> requested
>>> >> > 367500 Byte) (43602lx1,file13)
>>> >> > 20:45:08 HNDLR udp-tx
>>> >> > ------------------------------------------------------------
>>> >> > (43602lx1,file13)
>>> >> > 20:45:08 HNDLR udp-tx [ 3] local 192.168.1.100 port 60001
>>> >> > connected
>>> >> > with 192.168.1.42 port 60001 (43602lx1,file13)
>>> >> > 20:45:08 HNDLR udp-tx [ ID] Interval Transfer
>>> Bandwidth
>>> >> > (43602lx1,file13)
>>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.000-0.100 sec 11244030 Bytes
>>> >> 899522400
>>> >> > bits/sec (43602lx1,file13)
>>> >> > 20:45:08 LOG L1 attn 65
>>> >> > 20:45:08 INFO ::AF-A Setting attn to 65
>>> >> > 20:45:08 INFO Sleep 0.3 sec
>>> >> > 20:45:08 LOG itx-lx1 wl0.0: wlc_send_bar: seq 0x17f tid 0
>>> >> > 20:45:08 LOG itx-lx1 wl0.0: wlc_send_bar: seq 0x17f tid 0
>>> >> (repeated 1
>>> >> > times)
>>> >> > 20:45:08 LOG L1 attn 0
>>> >> > 20:45:08 INFO ::AF-A Setting attn to 0
>>> >> > 20:45:08 INFO Sleep 0.3 sec
>>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.100-0.200 sec 2625420 Bytes
>>> >> 210033600
>>> >> > bits/sec (43602lx1,file13)
>>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.200-0.300 sec 0.00 Bytes 0.00
>>> >> bits/sec
>>> >> > (43602lx1,file13)
>>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.300-0.400 sec 0.00 Bytes 0.00
>>> >> bits/sec
>>> >> > (43602lx1,file13)
>>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.400-0.500 sec 6251910 Bytes
>>> >> 500152800
>>> >> > bits/sec (43602lx1,file13)
>>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.500-0.600 sec 10406130 Bytes
>>> >> 832490400
>>> >> > bits/sec (43602lx1,file13)
>>> >> > 20:45:08 HNDLR udp-tx [ 3] 0.600-0.700 sec 10229730 Bytes
>>> >> 818378400
>>> >> > bits/sec (43602lx1,file13)
>>> >> > 20:45:08 INFO udp Kill signal -HUP sent to
>>> (43602lx1,file13) :
>>> >> > iperf -c 192.168.1.42 -w 367500 -l 1470 -u -b 2G -B 192.168.1.100
>>> >> > -i
>>> >> > 0.1
>>> >> > -fb -S 0x0 -T 255 -t 5356800 -p 60001
>>> >> > 20:45:09 INFO udp Kill signal -HUP sent to (sflx2,file12)
>>> >> > :
>>> >> iperf
>>> >> > -s -w 7350000 -l 1470 -u -i 0.1 -fb -p 60001
>>> >> > 20:45:09 HNDLR udp-tx Close actions for event handler
>>> >> > (43602lx1,file13)
>>> >> > 20:45:09 INFO udp-tx Closed : (43602lx1,file13)
>>> >> >
>>> >> > Bob
>>> >> > -----Original Message-----
>>> >> > From: Bob (Robert) McMahon
>>> >> > Sent: Wednesday, September 10, 2014 5:02 PM
>>> >> > To: Martin T; galla...@gmail.com
>>> >> > Cc: iperf-users@lists.sourceforge.net
>>> >> > Subject: RE: [Iperf-users] Iperf client sends out less UDP traffic
>>> than
>>> >> > determined with "-b" flag
>>> >> >
>>> >> > Hi Martin,
>>> >> >
>>> >> > Just to confirm Andrew, I'm pretty sure I've seen the write syscall
>>> on
>>> >> linux
>>> >> > return ENOBUFFs. I'll check it out in the next few days and let
>>> >> > you
>>> >> know.
>>> >> >
>>> >> >
>>> >> > Bob
>>> >> > -----Original Message-----
>>> >> > From: Martin T [mailto:m4rtn...@gmail.com]
>>> >> > Sent: Wednesday, September 10, 2014 5:12 AM
>>> >> > To: galla...@gmail.com; Bob (Robert) McMahon
>>> >> > Cc: iperf-users@lists.sourceforge.net
>>> >> > Subject: Re: [Iperf-users] Iperf client sends out less UDP traffic
>>> than
>>> >> > determined with "-b" flag
>>> >> >
>>> >> > Andrew, Bob:
>>> >> >
>>> >> > does Iperf use send() system call? Based on Iperf source code and
>>> >> > tests with strace, it seems to use write() system call to send data
>>> to
>>> >> > socket and according to write() manual page, it doesn't have a
>>> similar
>>> >> > error to ENOBUFS. However, I could easily be wrong here.
>>> >> >
>>> >> >
>>> >> > Andrew,
>>> >> >
>>> >> > if Linux kernel silently drops packets in case the device queue
>>> >> > overflows, then how does Iperf client know that? For example if I
>>> >> > start the Iperf client with "iperf -c 10.10.10.1 -fm -t 600 -i 60
>>> >> > -u
>>> >> > -b 500m" command and thus request Iperf to send data with bandwidth
>>> of
>>> >> > 500Mbps and Iperf client reports me that it has sent data at
>>> >> > 120Mbps
>>> >> > instead of 500Mbps:
>>> >> >
>>> >> > [ 3] 0.0-600.0 sec 8602 MBytes 120 Mbits/sec
>>> >> >
>>> >> > ..then this means that Iperf client somehow had to know that some
>>> >> > of
>>> >> > the data was dropped or Iperf client was restricted to send data at
>>> >> > faster rate than 120Mbps. How to explain this behavior?
>>> >> >
>>> >> >
>>> >> > thanks,
>>> >> > Martin
>>> >> >
>>> >> > On 9/8/14, Andrew Gallatin <galla...@gmail.com> wrote:
>>> >> >> No, just that it is fairly new (added 5 years ago in "6ce9e7b ip:
>>> >> >> Report
>>> >> >> qdisc packet drops"). Netperf did not have it either, until I
>>> >> >> tried
>>> >> >> to
>>> >> >> figure out why I was getting wildly inaccurate results from
>>> >> >> netperf
>>> >> >> (eg,
>>> >> >> seeing 20Gb/s on a 1GbE link).
>>> >> >>
>>> >> >>
>>> >> >> On Mon, Sep 8, 2014 at 11:34 AM, Bob (Robert) McMahon
>>> >> >> <rmcma...@broadcom.com
>>> >> >>> wrote:
>>> >> >>
>>> >> >>> Hi Drew,
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> Thanks for the detailed explanation. Do you know any reason that
>>> >> >>> iperf
>>> >> >>> doesn’t or shouldn’t set IP_RECVERR on the sending socket(s)?
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> Thanks,
>>> >> >>>
>>> >> >>> Bob
>>> >> >>>
>>> >> >>> *From:* Andrew Gallatin [mailto:galla...@gmail.com]
>>> >> >>> *Sent:* Monday, September 08, 2014 6:53 AM
>>> >> >>> *To:* Martin T
>>> >> >>> *Cc:* iperf-users
>>> >> >>> *Subject:* Re: [Iperf-users] Iperf client sends out less UDP
>>> traffic
>>> >> >>> than
>>> >> >>> determined with "-b" flag
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> Please, there are no "dropping" or "non dropping" NIC drivers,
>>> unless
>>> >> >>> some
>>> >> >>> driver is so broken that
>>> >> >>>
>>> >> >>> it silently returns NETDEV_TX_OK & does not stop its txqueue, but
>>> >> >>> a
>>> >> >>> driver
>>> >> >>> that broken would
>>> >> >>>
>>> >> >>> not be accepted into the kernel.
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> What is really happening is that the linux kernel is silently
>>> >> >>> dropping
>>> >> >>> packets when the device's
>>> >> >>>
>>> >> >>> queue is full *AND* when the device queue is too small to hold
>>> >> >>> the
>>> >> >>> entire
>>> >> >>> socket buffer's worth
>>> >> >>>
>>> >> >>> of traffic. If you want blocking behavior, you need to ensure
>>> >> >>> you
>>> >> >>> have
>>> >> >>> the driver's queue
>>> >> >>>
>>> >> >>> large enough (or the socket buffer small enough) so that the
>>> driver's
>>> >> >>> queue can hold an
>>> >> >>>
>>> >> >>> entire socket buffer worth of traffic (* the number of active
>>> sockets
>>> >> >>> per
>>> >> >>> queue). You can change
>>> >> >>>
>>> >> >>> the driver's queue size via ifconfig txqueuelen. It might be
>>> better
>>> >> >>> to
>>> >> >>> reduce the socketbuffer
>>> >> >>>
>>> >> >>> size (and remember that linux multiplies whatever value you give
>>> >> >>> it
>>> >> >>> by
>>> >> >>> 2,
>>> >> >>> to account for
>>> >> >>>
>>> >> >>> slop...).
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> Note that any silent drops are done by the kernel, not the NIC
>>> >> >>> and
>>> >> >>> that
>>> >> >>> this is
>>> >> >>>
>>> >> >>> (bizarrely) the expected behavior on linux. If you don't believe
>>> me,
>>> >> >>> read the send(2) man page on linux:
>>> >> >>>
>>> >> >>> ENOBUFS
>>> >> >>>
>>> >> >>> The output queue for a network interface was full.
>>> >> >>> This
>>> >> >>> generally indicates that the
>>> >> >>>
>>> >> >>> interface has stopped sending, but may be caused by
>>> >> >>> transient congestion. (Normally,
>>> >> >>>
>>> >> >>> this does not occur in Linux. Packets are just
>>> >> >>> silently
>>> >> >>> dropped when a device queue
>>> >> >>>
>>> >> >>> overflows.)
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> If iperf wants a correct accounting of the packets sent by the
>>> stack
>>> >> >>> vs
>>> >> >>> dropped, iperf must set
>>> >> >>>
>>> >> >>> the IP_RECVERR sockopt on the sending socket. This stops the
>>> kernel
>>> >> >>> from
>>> >> >>> silently
>>> >> >>>
>>> >> >>> dropping packets, and causes send() to return -ENOBUFS rather
>>> >> >>> than
>>> 0
>>> >> >>> when
>>> >> >>>
>>> >> >>> datagram is dropped due to lack of resources (eg, like BSD does).
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> Drew
>>> >> >>>
>>> >> >>
>>> >> >
>>> >>
>>> >
>>>
>>
>>
>
------------------------------------------------------------------------------
Want excitement?
Manually upgrade your production database.
When you want reliability, choose Perforce
Perforce version control. Predictably reliable.
http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk
_______________________________________________
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users
