Re: [Iperf-users] iperf3 and 100Gb

[Jeffrey Lane]

On Tue, Apr 17, 2018 at 3:09 PM, Bruce A. Mah  wrote:
> If memory serves me right, Jeffrey Lane wrote:
>> Hi all,
>>
>> I have what is kind of a silly question, but who has some experience
>> testing 100Gb with iperf3?
>>
>> I  just wanted to validate something with iperf3 to see if it is reasonable.
>>
>> With a single process running, the most I've been able to get out of a
>> 100Gb network port is a burst of about 65Gb/s with sustained averages
>> of around 50-55Gb/s.
>>
>> This is after a LOT of kernel tweaks, PCIe tweaks, and network config tweaks.
>>
>> So at this point, I'm thinking that what I'm seeing is a hardware
>> bottleneck, since iperf3 isn't multi-threaded.
>>
>> What I wanted to validate, to get around that is this:
>>
>> On the target side, I've kicked off four iperf3 processes all bound to
>> the same IP but listening on a different port.  Now, on the client
>> side, I kick off four iperf3 instances, one per remote port.  After 30
>> minutes of testing, each instance returns an average throughput of
>> about 23Gb/s.
>>
>> So in that scenario is it reasonable that 4 parallel threads reporting
>> 23Gb/s can be aggregated to assume we're actually seeing throughput of
>> 92Gb/s on the 100Gb port (thus nearly saturated)?
>
> That all seems reasonable.  We have some experience with iperf3 in very
> high speed links, and yes, there's some amount of tuning that might be
> involved.
>
> https://fasterdata.es.net/ has some information on tuning hardware and
> software.

Thanks, I'll look into it.

> Note that the reported throughput is application-level throughput
> (payload only) and doesn't include protocol overheads.

Yeah, I get that, which is why I don't expect to see numbers
approaching 100% of theoretical limit, I assumed there was a loss for
overhead somewhere.

Glad to know my math adds up, though, many thanks everyone!

>
> Bruce.
>
>
>
>
>



-- 
Jeff Lane
TPP / Server Certification Lead

"Entropy isn't what it used to be."

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

[Iperf-users] v3.5 works for UDP bitrate!!!

[Michael Fox]

I was able to compile and install v3.5 on Ubuntu (needed extra ldconfig) and
on Windows (using cygwin).  So far, the behavior for slower speeds (-b = 56K
to 400K bps) and various packet sizes (-l = 256 to 1440 bytes) seems to work
well whether sending from the client (Window) or the server (Ubuntu).

 

Thanks!

Michael

 

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

Re: [Iperf-users] iperf3 and 100Gb

[Bruce A. Mah]

If memory serves me right, Jeffrey Lane wrote:
> Hi all,
> 
> I have what is kind of a silly question, but who has some experience
> testing 100Gb with iperf3?
> 
> I  just wanted to validate something with iperf3 to see if it is reasonable.
> 
> With a single process running, the most I've been able to get out of a
> 100Gb network port is a burst of about 65Gb/s with sustained averages
> of around 50-55Gb/s.
> 
> This is after a LOT of kernel tweaks, PCIe tweaks, and network config tweaks.
> 
> So at this point, I'm thinking that what I'm seeing is a hardware
> bottleneck, since iperf3 isn't multi-threaded.
> 
> What I wanted to validate, to get around that is this:
> 
> On the target side, I've kicked off four iperf3 processes all bound to
> the same IP but listening on a different port.  Now, on the client
> side, I kick off four iperf3 instances, one per remote port.  After 30
> minutes of testing, each instance returns an average throughput of
> about 23Gb/s.
> 
> So in that scenario is it reasonable that 4 parallel threads reporting
> 23Gb/s can be aggregated to assume we're actually seeing throughput of
> 92Gb/s on the 100Gb port (thus nearly saturated)?

That all seems reasonable.  We have some experience with iperf3 in very
high speed links, and yes, there's some amount of tuning that might be
involved.

https://fasterdata.es.net/ has some information on tuning hardware and
software.

Note that the reported throughput is application-level throughput
(payload only) and doesn't include protocol overheads.

Bruce.







signature.asc
Description: OpenPGP digital signature
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

[Iperf-users] -l size = payload or complete packet

[Michael Fox]

Does the -l [KM] option specify just the payload size or the full
packet size (including headers)?

 

Thanks,

Michael

 

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

Re: [Iperf-users] Problem installing v3.5 on Ubuntu 16.04

[Bruce A. Mah]

If memory serves me right, Michael Fox wrote:
> I tried building and installing v3.5 on Ubuntu 16.04.  I normally use only
> packages.  So bear with me if I ask dumb questions:
> 
> I followed the instructions in the "INSTALL" file:
> ./configure   (no problems)
> make  (I saw a couple of warnings, but no errors)
> make check(no problems indicated)
> sudo make install (no observed problems)
> 
> So, then I tried to run it:
> 
> $ iperf3 -v
> iperf3: error while loading shared libraries: libiperf.so.0: cannot open
> shared object file: No such file or directory
> $
> 
> $ ls -l /usr/local/lib
> total 1456
> -rw-r--r-- 1 root root  952026 Apr 17 01:48 libiperf.a
> -rwxr-xr-x 1 root root 942 Apr 17 01:48 libiperf.la
> lrwxrwxrwx 1 root root  17 Apr 17 01:48 libiperf.so -> libiperf.so.0.0.0
> lrwxrwxrwx 1 root root  17 Apr 17 01:48 libiperf.so.0 ->
> libiperf.so.0.0.0
> -rwxr-xr-x 1 root root  523608 Apr 17 01:48 libiperf.so.0.0.0
> drwxrwsr-x 4 root staff   4096 Feb  2 11:37 python2.7
> drwxrwsr-x 3 root staff   4096 Dec 24 10:06 python3.5
> $
> 
> Any ideas?

Try running "ldconfig" as root to rebuild the shared library cache.

There's some more information about this in esnet/iperf#153:

https://github.com/esnet/iperf/issues/153

Bruce.





signature.asc
Description: OpenPGP digital signature
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

Re: [Iperf-users] Bandwidth option (-b) doesn't seem to work

[Bruce A. Mah]

If memory serves me right, Michael Fox wrote:

> How would I find the latest/current Windows binaries and Ubuntu packages? 
> 
> https://iperf.fr/iperf-download.php and
> https://iperf.fr/iperf-download.php#ubuntu-command only has up to 3.1.3

iperf.fr is a third-party Web site, so we (ESnet, principal developers
of iperf3) have no control over what content is hosted there.  In fact
it looks like the site hasn't been significantly updated in about two
years (judging by the versions of software hosted on it), and we've been
unsuccessful in contacting the maintainer of the site.

Because our development resources for maintaining iperf3 are limited, we
only provide source code.  It's up to OS distribution vendors or the
community to provide executables or packages.

Bruce.




signature.asc
Description: OpenPGP digital signature
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

Re: [Iperf-users] iperf3 and 100Gb

[Bob McMahon via Iperf-users]

Seems reasonable to me though I'm not an iperf3 expert.

Bob

PS.  As a side note, with iperf2
 which
does support multiple traffic threads, you can get 4 threads using -P 4.

On Tue, Apr 17, 2018 at 6:19 AM, Jeffrey Lane  wrote:

> Hi all,
>
> I have what is kind of a silly question, but who has some experience
> testing 100Gb with iperf3?
>
> I  just wanted to validate something with iperf3 to see if it is
> reasonable.
>
> With a single process running, the most I've been able to get out of a
> 100Gb network port is a burst of about 65Gb/s with sustained averages
> of around 50-55Gb/s.
>
> This is after a LOT of kernel tweaks, PCIe tweaks, and network config
> tweaks.
>
> So at this point, I'm thinking that what I'm seeing is a hardware
> bottleneck, since iperf3 isn't multi-threaded.
>
> What I wanted to validate, to get around that is this:
>
> On the target side, I've kicked off four iperf3 processes all bound to
> the same IP but listening on a different port.  Now, on the client
> side, I kick off four iperf3 instances, one per remote port.  After 30
> minutes of testing, each instance returns an average throughput of
> about 23Gb/s.
>
> So in that scenario is it reasonable that 4 parallel threads reporting
> 23Gb/s can be aggregated to assume we're actually seeing throughput of
> 92Gb/s on the 100Gb port (thus nearly saturated)?
>
> Thanks,
>
> Jeff
>
> --
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Iperf-users mailing list
> Iperf-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/iperf-users
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

[Iperf-users] Problem installing v3.5 on Ubuntu 16.04

[Michael Fox]

I tried building and installing v3.5 on Ubuntu 16.04.  I normally use only
packages.  So bear with me if I ask dumb questions:

I followed the instructions in the "INSTALL" file:
./configure (no problems)
make(I saw a couple of warnings, but no errors)
make check  (no problems indicated)
sudo make install   (no observed problems)

So, then I tried to run it:

$ iperf3 -v
iperf3: error while loading shared libraries: libiperf.so.0: cannot open
shared object file: No such file or directory
$

$ ls -l /usr/local/lib
total 1456
-rw-r--r-- 1 root root  952026 Apr 17 01:48 libiperf.a
-rwxr-xr-x 1 root root 942 Apr 17 01:48 libiperf.la
lrwxrwxrwx 1 root root  17 Apr 17 01:48 libiperf.so -> libiperf.so.0.0.0
lrwxrwxrwx 1 root root  17 Apr 17 01:48 libiperf.so.0 ->
libiperf.so.0.0.0
-rwxr-xr-x 1 root root  523608 Apr 17 01:48 libiperf.so.0.0.0
drwxrwsr-x 4 root staff   4096 Feb  2 11:37 python2.7
drwxrwsr-x 3 root staff   4096 Dec 24 10:06 python3.5
$

Any ideas?

Michael



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

Re: [Iperf-users] Having problems with specifying different network interface for iperf3

[Marcel]

It seems that I still have much to learn.

I uses the wifi as default network and that's why the modem couldn't 
connect!


Changing the modem to the default network solved the problem and it now 
works as expected.


Sorry for bothering you guys and thanks for your help!


Am 16.04.2018 um 23:20 schrieb Bruce A. Mah:

If memory serves me right, Marcel wrote:

Hey Bruce,

thanks for the fast reply.

Actually I'm using a Python script and therefor I get the IP of the
interface using netifaces.

I also checked that using ifconfig while running the script and netstat
-an showed the same IP.

Here was the output (using the IPs from before) while running the iperf3
command:

tcp    0  1 12.34.56.78:12345  12.12.12.12:  SYN_SENT

I can't see what might be wrong here, do you have an idea where else I
could look for an issue?

Hmmm...that looks pretty much normal to me...this is indeed more than a
little odd.  It's weird that if you use "-B" to specify the outgoing
address, it stops working even if you specify the same address the
routing table would have picked anyway.  At the moment I can't think of
anything to try next...I know that's not much help.

Bruce.


Am 13.04.2018 um 20:25 schrieb Bruce A. Mah:

If memory serves me right, Marcel wrote:


I'm trying to do an iperf3 test using a Huawei ME909 wwan modem on a
raspberry pi zero w.

I use the command as iperf3 -c 12.12.12.12 -t 3 -p 5201 -B 12.34.56.78

where 12.12.12.12 is the ip of the server machine and 12.34.56.78 is the
ip of the Huawei modem.

With this command, I get a timeout error, but when I ommit the -B part
with the ip of the modem, iperf3 uses the wlan0 interface and behaves
normally.

I should also mention that I connected the modem with AT commands
(AT+COPS, AT^NDISDUP) and when I do a ping over that interface it also
behaves as expected, but when I use a package size over 1300 for ping, I
also get a timeout error.

I'm quite new to this kind of stuff, so I would appreciate any help!

Hi Marcel--

Hmmm...I don't completely understand the networking setup, but the
argument to -B is supposed to be the address of the network interface on
the iperf3 client machine that you want to bind to.  So you're saying
the Huawei modem presents an IP network interface with the address
12.34.56.78 (I presume that's not the real address, which is fine).

Any chance you can do "ip addr" (or the equivalent) for this interface?
I'd like to check if the address assigned is really the same as what
you're giving in the "-B" parameter to iperf3.  Also you can see the MTU
for the link (might help with the ping issue you mentioned).

While the test is running you might be able to (from another window) use
"netstat -an" as additional verification of the local address being used.

Bruce.



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot



___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users






--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users