Re: Private: Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

[Florin Coras]

As mentioned previously, that is not supported for connects currently.  

Regards,
Florin

> On Mar 31, 2022, at 9:44 PM, weizhen9...@163.com wrote:
> 
> Hi,
> I describe our scene in detail. We use nginx in vpp host stack as a proxy. 
> And we add some features in nginx. For example, nginx close upstream tcp 
> links actively and this causes a lot of TIME_WAIT states in nginx proxy when 
> we test the performance of nginx using vpp host stack. So we configure kernel 
> parameters to multiplex tcp port and recycle port quickly. But I can't find 
> vpp host stack parameters which is similar with kernel parameters.
> How can we configure vpp parameters to reuse tcp port and recycle port 
> quickly?
> Thanks.  
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21177): https://lists.fd.io/g/vpp-dev/message/21177
Mute This Topic: https://lists.fd.io/mt/90157992/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: Private: Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

[weizhen9612]

Hi,
I describe our scene in detail. We use nginx in vpp host stack as a proxy. And 
we add some features in nginx. For example, nginx close upstream tcp links 
actively and this causes a lot of TIME_WAIT states in nginx proxy when we test 
the performance of nginx using vpp host stack. So we configure kernel 
parameters to multiplex tcp port and recycle port quickly. But I can't find vpp 
host stack parameters which is similar with kernel parameters.
How can we configure vpp parameters to reuse tcp port and recycle port quickly?
Thanks.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21176): https://lists.fd.io/g/vpp-dev/message/21176
Mute This Topic: https://lists.fd.io/mt/90157992/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: Private: Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

[Florin Coras]

Hi,

Given that 20k connections are being actively opened, those on main thread, and 
40k are established, those on the workers, suggests that tcp runs out of ports 
for connects. If possible either increase the number of destination IPs for 
nginx or try “tcp src-address ip1-ip2” and pass in a range of consecutive ips 
that could be used the vpp as source ips.   

The two workers seem to be pretty lightly loaded since loops/2 is over 2M, 
i.e., they’re mostly spinning empty, without work. 

Regards,
Florin

> On Mar 31, 2022, at 8:21 PM, weizhen9...@163.com wrote:
> 
> Hi,
> 
> From this test results, can you see some errors? Why is the performance of 
> nginx using vpp low?
> 
> Thanks. 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21175): https://lists.fd.io/g/vpp-dev/message/21175
Mute This Topic: https://lists.fd.io/mt/90157992/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: Private: Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

[weizhen9612]

Hi,

>From this test results, can you see some errors? Why is the performance of 
>nginx using vpp low?

Thanks.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21174): https://lists.fd.io/g/vpp-dev/message/21174
Mute This Topic: https://lists.fd.io/mt/90157992/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: Private: Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

[Florin Coras]

TCP accepts connections in time-wait (see here [1]) but won’t reuse ports of 
connection in time-wait for connects. If you expect lots of active opens from 
nginx to only one destination ip and you have multiple source ips, you could 
try to use "tcp src-address” cli. 

Regards,
Florin

[1] https://git.fd.io/vpp/tree/src/vnet/tcp/tcp_input.c#n2667

> On Mar 31, 2022, at 7:00 PM, weizhen9...@163.com wrote:
> 
> Hi, 
> 
> When we test the performance of nginx using vpp host stack, we execute the 
> following command.
> #show run 
> 
> 
> 
> 
> 
> #show session
> 
> 
> 
> Besides, we add some new features in nginx and this feature will cause a lot 
> of time_wait states. In order to reuse TIME_WAIT state tcp_tw_reuse and 
> tcp_tw_recycle. Besides, to make the port unrestricted, we modify other 
> parameters.
> So how can we modify the vpp host stack parameters?
> Thanks.
> 
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21173): https://lists.fd.io/g/vpp-dev/message/21173
Mute This Topic: https://lists.fd.io/mt/90157992/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: Private: Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

[Florin Coras]

I didn’t mean you should switch to envoy, just that throughput is pretty low 
probably because of some configuration. What that configuration is is not 
obvious unfortunately. 

Regarding the kernel parameters, we have time wait reuse enabled (equivalent to 
tcp_tw_reuse) but that should not matter unless nginx establishes a new 
connection for each request. Does it, as part of the features you’ve added?

Could you do: 
- clear run; show run - in vpp when under load
- show sessions - to see the number of session in vpp 

Also, could you reduce the number of workers to maybe 1-2 in nginx and 1 in 
vpp, to get some base performance readings?

Regards,
Florin 

[1] https://wiki.fd.io/images/0/08/Using_vpp_as_envoys_network_stack.pdf 


> On Mar 31, 2022, at 8:56 AM, weizhen9...@163.com wrote:
> 
> Hi,
> 
> I'm doing proxying with nginx. And we develop some new functions in nginx. 
> The performance of nginx in kernel host stack is higher than nginx using vpp 
> host stack. When testing the nginx in kernel host stack, we modify the kernel 
> parameters in kernel host stack. When using vpp host stack, how can we modify 
> the similar parameters?
> 
> Besides, we don't use the envoy. We develop some functions in nginx.
>> 
>> Hi, 
>> 
>> Spoke too soon. Noticed you’re doing proxying with nginx. 
>> 
>> What does clear run; show run report in vpp when under load?
>> 
>> Side note, with envoy I’m seeing much better numbers. See for instance slide 
>> 12 here [1]. So I suspect this is a configuration issue but can’t tell 
>> upfront what it is.
>> 
>> Regards,
>> Florin

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21168): https://lists.fd.io/g/vpp-dev/message/21168
Mute This Topic: https://lists.fd.io/mt/90157992/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-