Re: [Iperf-users] iperf 3.1.3 TCP throughput issue

[Bob McMahon via Iperf-users]

The tcptrace tool may provide insights.  Also, if you
can test without -R then a run with iperf2
and -e could provide confirmation
of the expected throughputs and possibly some other information, e.g. the
RTT samples.  Also, with a 0.005 s sample interval one can get insights
into the TCP "ramp up."

Bob

On Sat, May 12, 2018 at 10:17 AM, Kaliaperumal, Rajesh <
rajesh.kaliaperu...@arris.com> wrote:

> Thanks Bruce. I had to do MSS mangling to avoid fragmentation of packets
> after ESP header addition.
>
> With single streams I get about 1/3rd of what I get using multiple streams
>
> Thanks and regards
> Rajesh
>
>
> > On May 12, 2018, at 2:34 AM, Bruce A. Mah  wrote:
> >
> > If memory serves me right, Kaliaperumal, Rajesh wrote:
> >> Hi Bruce,
> >>Single stream was pretty low and then I started trying multiple
> streams. I am expecting a throughput of the order of 70-100Mbps (my ISP DL
> being the limiting factor).
> >
> > Hmmm.  I don't have any good explanation off-hand.  The only two
> > thoughts I can think of right now are:  1) the IPsec tunnel and MSS
> > manipulation are unusual features of the path, and 2) that's a fairly
> > old version of iperf3.  I'm not sure if either of those are a factor,
> > just saying they stand out to me.
> >
> > Note that using multiple streams isn't necessarily going to scale the
> > throughput linearly.
> >
> > Bruce.
> >
> >> On 5/9/18, 4:02 PM, "Bruce A. Mah"  wrote:
> >>
> >>If memory serves me right, Kaliaperumal, Rajesh wrote:
> >>> Hi all,
> >>>
> >>>
> >>>
> >>> Iperf version – iperf 3.1.3
> >>>
> >>>
> >>>
> >>> My set up:  iperf server (GCE) <=> Gateway VM (GCE) <=> IPsec server
> >>> (GCE) <=> WAN <=> Laptop with IPSec Client
> >>>
> >>>
> >>>
> >>> *My test:*
> >>>
> >>> Server – /iperf3 -s /
> >>>
> >>> Client – /iperf3 -c  -P 15 -R -w 768k/
> >>>
> >>> Due to IPsec header, my Gateway VM modifies the MSS size of the
> >>> SYN-SYN/ACK message so that the TCP segments are not fragmented after
> >>> IPSec ESP encryption.
> >>>
> >>>
> >>>
> >>> *My Observation:*
> >>>
> >>> When I perform a downlink test by using the “-R” option at the iperf
> >>> client side, my throughput has a lot of variation and quickly drops to
> 1
> >>> or 2 Mbps. In other words, the iPerf server reduces the rate at which
> it
> >>> pumps data. There are no packet drops on other VMs in my setup.
> >>>
> >>>
> >>>
> >>> I want to understand how iperf server determines the rate at which it
> >>> has to pump data?
> >>
> >>With those parameters (and thank you for providing that information),
> >>the server basically just sends data as fast as it can on the 15 TCP
> >>connections you specified by the -P parameter.  The speed at which it
> >>can send data is governed by the TCP congestion control and flow
> control
> >>algorithms.
> >>
> >>Of course those TCP connections will interact with each other, and
> any
> >>other network traffic along the path.  If the system isn't behaving
> in a
> >>way that makes sense to you, might I suggest trying your experiment
> with
> >>just a single TCP connection?  Also, what throughput were you
> expecting
> >>to get?
> >>
> >>Bruce.
> >>
> >>
> >>
> >>
> >
> >
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Iperf-users mailing list
> Iperf-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/iperf-users
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

Re: [Iperf-users] iperf 3.1.3 TCP throughput issue

[Kaliaperumal, Rajesh]

Thanks Bruce. I had to do MSS mangling to avoid fragmentation of packets after 
ESP header addition. 

With single streams I get about 1/3rd of what I get using multiple streams 

Thanks and regards 
Rajesh 


> On May 12, 2018, at 2:34 AM, Bruce A. Mah  wrote:
> 
> If memory serves me right, Kaliaperumal, Rajesh wrote:
>> Hi Bruce,
>>Single stream was pretty low and then I started trying multiple streams. 
>> I am expecting a throughput of the order of 70-100Mbps (my ISP DL being the 
>> limiting factor).
> 
> Hmmm.  I don't have any good explanation off-hand.  The only two
> thoughts I can think of right now are:  1) the IPsec tunnel and MSS
> manipulation are unusual features of the path, and 2) that's a fairly
> old version of iperf3.  I'm not sure if either of those are a factor,
> just saying they stand out to me.
> 
> Note that using multiple streams isn't necessarily going to scale the
> throughput linearly.
> 
> Bruce.
> 
>> On 5/9/18, 4:02 PM, "Bruce A. Mah"  wrote:
>> 
>>If memory serves me right, Kaliaperumal, Rajesh wrote:
>>> Hi all,
>>> 
>>> 
>>> 
>>> Iperf version – iperf 3.1.3
>>> 
>>> 
>>> 
>>> My set up:  iperf server (GCE) <=> Gateway VM (GCE) <=> IPsec server
>>> (GCE) <=> WAN <=> Laptop with IPSec Client
>>> 
>>> 
>>> 
>>> *My test:*
>>> 
>>> Server – /iperf3 -s /
>>> 
>>> Client – /iperf3 -c  -P 15 -R -w 768k/
>>> 
>>> Due to IPsec header, my Gateway VM modifies the MSS size of the
>>> SYN-SYN/ACK message so that the TCP segments are not fragmented after
>>> IPSec ESP encryption.
>>> 
>>> 
>>> 
>>> *My Observation:*
>>> 
>>> When I perform a downlink test by using the “-R” option at the iperf
>>> client side, my throughput has a lot of variation and quickly drops to 1
>>> or 2 Mbps. In other words, the iPerf server reduces the rate at which it
>>> pumps data. There are no packet drops on other VMs in my setup.
>>> 
>>> 
>>> 
>>> I want to understand how iperf server determines the rate at which it
>>> has to pump data?
>> 
>>With those parameters (and thank you for providing that information),
>>the server basically just sends data as fast as it can on the 15 TCP
>>connections you specified by the -P parameter.  The speed at which it
>>can send data is governed by the TCP congestion control and flow control
>>algorithms.
>> 
>>Of course those TCP connections will interact with each other, and any
>>other network traffic along the path.  If the system isn't behaving in a
>>way that makes sense to you, might I suggest trying your experiment with
>>just a single TCP connection?  Also, what throughput were you expecting
>>to get?
>> 
>>Bruce.
>> 
>> 
>> 
>> 
> 
> 
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

Re: [Iperf-users] iperf 3.1.3 TCP throughput issue

[Bruce A. Mah]

If memory serves me right, Kaliaperumal, Rajesh wrote:
> Hi Bruce,
>   Single stream was pretty low and then I started trying multiple 
> streams. I am expecting a throughput of the order of 70-100Mbps (my ISP DL 
> being the limiting factor).

Hmmm.  I don't have any good explanation off-hand.  The only two
thoughts I can think of right now are:  1) the IPsec tunnel and MSS
manipulation are unusual features of the path, and 2) that's a fairly
old version of iperf3.  I'm not sure if either of those are a factor,
just saying they stand out to me.

Note that using multiple streams isn't necessarily going to scale the
throughput linearly.

Bruce.

> On 5/9/18, 4:02 PM, "Bruce A. Mah"  wrote:
> 
> If memory serves me right, Kaliaperumal, Rajesh wrote:
> > Hi all,
> > 
> >  
> > 
> > Iperf version – iperf 3.1.3
> > 
> >  
> > 
> > My set up:  iperf server (GCE) <=> Gateway VM (GCE) <=> IPsec server
> > (GCE) <=> WAN <=> Laptop with IPSec Client
> > 
> >  
> > 
> > *My test:*
> > 
> > Server – /iperf3 -s /
> > 
> > Client – /iperf3 -c  -P 15 -R -w 768k/
> > 
> > Due to IPsec header, my Gateway VM modifies the MSS size of the
> > SYN-SYN/ACK message so that the TCP segments are not fragmented after
> > IPSec ESP encryption.
> > 
> >  
> > 
> > *My Observation:*
> > 
> > When I perform a downlink test by using the “-R” option at the iperf
> > client side, my throughput has a lot of variation and quickly drops to 1
> > or 2 Mbps. In other words, the iPerf server reduces the rate at which it
> > pumps data. There are no packet drops on other VMs in my setup.
> > 
> >  
> > 
> > I want to understand how iperf server determines the rate at which it
> > has to pump data?
> 
> With those parameters (and thank you for providing that information),
> the server basically just sends data as fast as it can on the 15 TCP
> connections you specified by the -P parameter.  The speed at which it
> can send data is governed by the TCP congestion control and flow control
> algorithms.
> 
> Of course those TCP connections will interact with each other, and any
> other network traffic along the path.  If the system isn't behaving in a
> way that makes sense to you, might I suggest trying your experiment with
> just a single TCP connection?  Also, what throughput were you expecting
> to get?
> 
> Bruce.
> 
> 
> 
> 




signature.asc
Description: OpenPGP digital signature
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

Re: [Iperf-users] iperf 3.1.3 TCP throughput issue

[Kaliaperumal, Rajesh]

Hi Bruce,
Single stream was pretty low and then I started trying multiple 
streams. I am expecting a throughput of the order of 70-100Mbps (my ISP DL 
being the limiting factor).

Thanks and regards,
Rajesh K


On 5/9/18, 4:02 PM, "Bruce A. Mah"  wrote:

If memory serves me right, Kaliaperumal, Rajesh wrote:
> Hi all,
> 
>  
> 
> Iperf version – iperf 3.1.3
> 
>  
> 
> My set up:  iperf server (GCE) <=> Gateway VM (GCE) <=> IPsec server
> (GCE) <=> WAN <=> Laptop with IPSec Client
> 
>  
> 
> *My test:*
> 
> Server – /iperf3 -s /
> 
> Client – /iperf3 -c  -P 15 -R -w 768k/
> 
> Due to IPsec header, my Gateway VM modifies the MSS size of the
> SYN-SYN/ACK message so that the TCP segments are not fragmented after
> IPSec ESP encryption.
> 
>  
> 
> *My Observation:*
> 
> When I perform a downlink test by using the “-R” option at the iperf
> client side, my throughput has a lot of variation and quickly drops to 1
> or 2 Mbps. In other words, the iPerf server reduces the rate at which it
> pumps data. There are no packet drops on other VMs in my setup.
> 
>  
> 
> I want to understand how iperf server determines the rate at which it
> has to pump data?

With those parameters (and thank you for providing that information),
the server basically just sends data as fast as it can on the 15 TCP
connections you specified by the -P parameter.  The speed at which it
can send data is governed by the TCP congestion control and flow control
algorithms.

Of course those TCP connections will interact with each other, and any
other network traffic along the path.  If the system isn't behaving in a
way that makes sense to you, might I suggest trying your experiment with
just a single TCP connection?  Also, what throughput were you expecting
to get?

Bruce.




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

Re: [Iperf-users] iperf 3.1.3 TCP throughput issue

[Bruce A. Mah]

If memory serves me right, Kaliaperumal, Rajesh wrote:
> Hi all,
> 
>  
> 
> Iperf version – iperf 3.1.3
> 
>  
> 
> My set up:  iperf server (GCE) <=> Gateway VM (GCE) <=> IPsec server
> (GCE) <=> WAN <=> Laptop with IPSec Client
> 
>  
> 
> *My test:*
> 
> Server – /iperf3 -s /
> 
> Client – /iperf3 -c  -P 15 -R -w 768k/
> 
> Due to IPsec header, my Gateway VM modifies the MSS size of the
> SYN-SYN/ACK message so that the TCP segments are not fragmented after
> IPSec ESP encryption.
> 
>  
> 
> *My Observation:*
> 
> When I perform a downlink test by using the “-R” option at the iperf
> client side, my throughput has a lot of variation and quickly drops to 1
> or 2 Mbps. In other words, the iPerf server reduces the rate at which it
> pumps data. There are no packet drops on other VMs in my setup.
> 
>  
> 
> I want to understand how iperf server determines the rate at which it
> has to pump data?

With those parameters (and thank you for providing that information),
the server basically just sends data as fast as it can on the 15 TCP
connections you specified by the -P parameter.  The speed at which it
can send data is governed by the TCP congestion control and flow control
algorithms.

Of course those TCP connections will interact with each other, and any
other network traffic along the path.  If the system isn't behaving in a
way that makes sense to you, might I suggest trying your experiment with
just a single TCP connection?  Also, what throughput were you expecting
to get?

Bruce.




signature.asc
Description: OpenPGP digital signature
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users

[Iperf-users] iperf 3.1.3 TCP throughput issue

[Kaliaperumal, Rajesh]

Hi all,

Iperf version – iperf 3.1.3

My set up:  iperf server (GCE) <=> Gateway VM (GCE) <=> IPsec server (GCE) <=> 
WAN <=> Laptop with IPSec Client

My test:
Server – iperf3 -s
Client – iperf3 -c  -P 15 -R -w 768k
Due to IPsec header, my Gateway VM modifies the MSS size of the SYN-SYN/ACK 
message so that the TCP segments are not fragmented after IPSec ESP encryption.

My Observation:
When I perform a downlink test by using the “-R” option at the iperf client 
side, my throughput has a lot of variation and quickly drops to 1 or 2 Mbps. In 
other words, the iPerf server reduces the rate at which it pumps data. There 
are no packet drops on other VMs in my setup.

I want to understand how iperf server determines the rate at which it has to 
pump data?

Thanks and regards,
Rajesh K

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Iperf-users mailing list
Iperf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users