In my (cursory) read of the 3.x code, it looked like it
ignores NET_SOFTERROR returns & retries. It would be nice if it reported
the amount of ENOBUF errors it received like netperf does..
On Wed, Sep 17, 2014 at 4:09 AM, Martin T <m4rtn...@gmail.com> wrote:
> Andrew, Bob,
>
> did I understand you correctly that while ENOBUFS error code is not
> shown in Iperf client output, the Iperf client actually takes it into
> account and thus sends data out less frequently?
>
>
> regards,
> Martin
>
> On 9/11/14, Bob (Robert) McMahon <rmcma...@broadcom.com> wrote:
> > I thought I remembered hitting the ENOBUFs which is the case per the
> > source.
> >
> > // perform write
> > currLen = write( mSettings->mSock, mBuf, mSettings->mBufLen );
> > if ( currLen < 0 && errno != ENOBUFS ) {
> > WARN_errno( currLen < 0, "write2" );
> > break;
> > }
> > // report packets
> > reportstruct->packetLen = currLen;
> > ReportPacket( mSettings->reporthdr, reportstruct );
> >
> > A minor nit, the currLen may need to set to zero when errno==ENOBUFFS
> before
> > the ReportPacket()
> >
> > Also, ione could count the number of writes that return ENOBUFFS but not
> > sure why that's interesting.
> >
> > As an FYI, I have a tool that manages multiple, distributed iperf
> sessions
> > which I use for wi-fi testing. The following shows an 802.11 AC wi-fi
> link
> > that maxes out around 800Mbs. The offered load is 2G and the iperf
> client
> > seems to be reporting properly and responding as expected when RF
> > attenuation is added and then removed.
> >
> > utf> udp start; udp linkcheck -now; L1 attn 65; UTF::Sleep 0.3; L1 attn
> 0;
> > UTF::Sleep 0.3; udp stop
> > 20:45:08 LOG sfast-lx2 iperf -s -w 7350000 -l 1470 -u -i 0.1 -fb -p
> > 60001
> > 20:45:08 HNDLR_ udp-rx Server listening on UDP port 60001
> > (sflx2,file12)
> > 20:45:08 HNDLR_ udp-rx Receiving 1470 byte datagrams (sflx2,file12)
> > 20:45:08 HNDLR_ udp-rx UDP buffer size: 8388608 Byte (WARNING:
> > requested 7350000 Byte) (sflx2,file12)
> > 20:45:08 LOG itx-lx1 iperf -c 192.168.1.42 -w 367500 -l 1470 -u -b
> 2G
> > -B 192.168.1.100 -i 0.1 -fb -S 0x0 -T 255 -t 5356800 -p 60001
> > 20:45:08 HNDLR udp-tx
> > ------------------------------------------------------------
> > (43602lx1,file13)
> > 20:45:08 HNDLR_ udp-rx [ 3] local 192.168.1.42 port 60001 connected
> > with 192.168.1.100 port 60001 (sflx2,file12)
> > 20:45:08 HNDLR udp-tx Client connecting to 192.168.1.42, UDP port
> 60001
> > (43602lx1,file13)
> > 20:45:08 INFO udp-tx TX start: protocol=UDP Offered rate=2G
> > pktsize=1470 delay=5us
> > 20:45:08 HNDLR udp-tx Binding to local address 192.168.1.100
> > (43602lx1,file13)
> > 20:45:08 HNDLR udp-tx Sending 1470 byte datagrams (43602lx1,file13)
> > 20:45:08 HNDLR udp-tx UDP buffer size: 735000 Byte (WARNING:
> requested
> > 367500 Byte) (43602lx1,file13)
> > 20:45:08 HNDLR udp-tx
> > ------------------------------------------------------------
> > (43602lx1,file13)
> > 20:45:08 HNDLR udp-tx [ 3] local 192.168.1.100 port 60001 connected
> > with 192.168.1.42 port 60001 (43602lx1,file13)
> > 20:45:08 HNDLR udp-tx [ ID] Interval Transfer Bandwidth
> > (43602lx1,file13)
> > 20:45:08 HNDLR udp-tx [ 3] 0.000-0.100 sec 11244030 Bytes
> 899522400
> > bits/sec (43602lx1,file13)
> > 20:45:08 LOG L1 attn 65
> > 20:45:08 INFO ::AF-A Setting attn to 65
> > 20:45:08 INFO Sleep 0.3 sec
> > 20:45:08 LOG itx-lx1 wl0.0: wlc_send_bar: seq 0x17f tid 0
> > 20:45:08 LOG itx-lx1 wl0.0: wlc_send_bar: seq 0x17f tid 0
> (repeated 1
> > times)
> > 20:45:08 LOG L1 attn 0
> > 20:45:08 INFO ::AF-A Setting attn to 0
> > 20:45:08 INFO Sleep 0.3 sec
> > 20:45:08 HNDLR udp-tx [ 3] 0.100-0.200 sec 2625420 Bytes
> 210033600
> > bits/sec (43602lx1,file13)
> > 20:45:08 HNDLR udp-tx [ 3] 0.200-0.300 sec 0.00 Bytes 0.00
> bits/sec
> > (43602lx1,file13)
> > 20:45:08 HNDLR udp-tx [ 3] 0.300-0.400 sec 0.00 Bytes 0.00
> bits/sec
> > (43602lx1,file13)
> > 20:45:08 HNDLR udp-tx [ 3] 0.400-0.500 sec 6251910 Bytes
> 500152800
> > bits/sec (43602lx1,file13)
> > 20:45:08 HNDLR udp-tx [ 3] 0.500-0.600 sec 10406130 Bytes
> 832490400
> > bits/sec (43602lx1,file13)
> > 20:45:08 HNDLR udp-tx [ 3] 0.600-0.700 sec 10229730 Bytes
> 818378400
> > bits/sec (43602lx1,file13)
> > 20:45:08 INFO udp Kill signal -HUP sent to (43602lx1,file13) :
> > iperf -c 192.168.1.42 -w 367500 -l 1470 -u -b 2G -B 192.168.1.100 -i 0.1
> > -fb -S 0x0 -T 255 -t 5356800 -p 60001
> > 20:45:09 INFO udp Kill signal -HUP sent to (sflx2,file12) :
> iperf
> > -s -w 7350000 -l 1470 -u -i 0.1 -fb -p 60001
> > 20:45:09 HNDLR udp-tx Close actions for event handler
> > (43602lx1,file13)
> > 20:45:09 INFO udp-tx Closed : (43602lx1,file13)
> >
> > Bob
> > -----Original Message-----
> > From: Bob (Robert) McMahon
> > Sent: Wednesday, September 10, 2014 5:02 PM
> > To: Martin T; galla...@gmail.com
> > Cc: iperf-users@lists.sourceforge.net
> > Subject: RE: [Iperf-users] Iperf client sends out less UDP traffic than
> > determined with "-b" flag
> >
> > Hi Martin,
> >
> > Just to confirm Andrew, I'm pretty sure I've seen the write syscall on
> linux
> > return ENOBUFFs. I'll check it out in the next few days and let you
> know.
> >
> >
> > Bob
> > -----Original Message-----
> > From: Martin T [mailto:m4rtn...@gmail.com]
> > Sent: Wednesday, September 10, 2014 5:12 AM
> > To: galla...@gmail.com; Bob (Robert) McMahon
> > Cc: iperf-users@lists.sourceforge.net
> > Subject: Re: [Iperf-users] Iperf client sends out less UDP traffic than
> > determined with "-b" flag
> >
> > Andrew, Bob:
> >
> > does Iperf use send() system call? Based on Iperf source code and
> > tests with strace, it seems to use write() system call to send data to
> > socket and according to write() manual page, it doesn't have a similar
> > error to ENOBUFS. However, I could easily be wrong here.
> >
> >
> > Andrew,
> >
> > if Linux kernel silently drops packets in case the device queue
> > overflows, then how does Iperf client know that? For example if I
> > start the Iperf client with "iperf -c 10.10.10.1 -fm -t 600 -i 60 -u
> > -b 500m" command and thus request Iperf to send data with bandwidth of
> > 500Mbps and Iperf client reports me that it has sent data at 120Mbps
> > instead of 500Mbps:
> >
> > [ 3] 0.0-600.0 sec 8602 MBytes 120 Mbits/sec
> >
> > ..then this means that Iperf client somehow had to know that some of
> > the data was dropped or Iperf client was restricted to send data at
> > faster rate than 120Mbps. How to explain this behavior?
> >
> >
> > thanks,
> > Martin
> >
> > On 9/8/14, Andrew Gallatin <galla...@gmail.com> wrote:
> >> No, just that it is fairly new (added 5 years ago in "6ce9e7b ip: Report
> >> qdisc packet drops"). Netperf did not have it either, until I tried to
> >> figure out why I was getting wildly inaccurate results from netperf (eg,
> >> seeing 20Gb/s on a 1GbE link).
> >>
> >>
> >> On Mon, Sep 8, 2014 at 11:34 AM, Bob (Robert) McMahon
> >> <rmcma...@broadcom.com
> >>> wrote:
> >>
> >>> Hi Drew,
> >>>
> >>>
> >>>
> >>> Thanks for the detailed explanation. Do you know any reason that iperf
> >>> doesn’t or shouldn’t set IP_RECVERR on the sending socket(s)?
> >>>
> >>>
> >>>
> >>> Thanks,
> >>>
> >>> Bob
> >>>
> >>> *From:* Andrew Gallatin [mailto:galla...@gmail.com]
> >>> *Sent:* Monday, September 08, 2014 6:53 AM
> >>> *To:* Martin T
> >>> *Cc:* iperf-users
> >>> *Subject:* Re: [Iperf-users] Iperf client sends out less UDP traffic
> >>> than
> >>> determined with "-b" flag
> >>>
> >>>
> >>>
> >>> Please, there are no "dropping" or "non dropping" NIC drivers, unless
> >>> some
> >>> driver is so broken that
> >>>
> >>> it silently returns NETDEV_TX_OK & does not stop its txqueue, but a
> >>> driver
> >>> that broken would
> >>>
> >>> not be accepted into the kernel.
> >>>
> >>>
> >>>
> >>> What is really happening is that the linux kernel is silently dropping
> >>> packets when the device's
> >>>
> >>> queue is full *AND* when the device queue is too small to hold the
> >>> entire
> >>> socket buffer's worth
> >>>
> >>> of traffic. If you want blocking behavior, you need to ensure you have
> >>> the driver's queue
> >>>
> >>> large enough (or the socket buffer small enough) so that the driver's
> >>> queue can hold an
> >>>
> >>> entire socket buffer worth of traffic (* the number of active sockets
> >>> per
> >>> queue). You can change
> >>>
> >>> the driver's queue size via ifconfig txqueuelen. It might be better to
> >>> reduce the socketbuffer
> >>>
> >>> size (and remember that linux multiplies whatever value you give it by
> >>> 2,
> >>> to account for
> >>>
> >>> slop...).
> >>>
> >>>
> >>>
> >>> Note that any silent drops are done by the kernel, not the NIC and that
> >>> this is
> >>>
> >>> (bizarrely) the expected behavior on linux. If you don't believe me,
> >>> read the send(2) man page on linux:
> >>>
> >>> ENOBUFS
> >>>
> >>> The output queue for a network interface was full. This
> >>> generally indicates that the
> >>>
> >>> interface has stopped sending, but may be caused by
> >>> transient congestion. (Normally,
> >>>
> >>> this does not occur in Linux. Packets are just silently
> >>> dropped when a device queue
> >>>
> >>> overflows.)
> >>>
> >>>
> >>>
> >>> If iperf wants a correct accounting of the packets sent by the stack vs
> >>> dropped, iperf must set
> >>>
> >>> the IP_RECVERR sockopt on the sending socket. This stops the kernel
> >>> from
> >>> silently
> >>>
> >>> dropping packets, and causes send() to return -ENOBUFS rather than 0
> >>> when
> >>>
> >>> datagram is dropped due to lack of resources (eg, like BSD does).
> >>>
> >>>
> >>>
> >>> Drew
> >>>
> >>
> >
>
------------------------------------------------------------------------------
Want excitement?
Manually upgrade your production database.
When you want reliability, choose Perforce
Perforce version control. Predictably reliable.
http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk
_______________________________________________
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users
