Re: Private: Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

2022-03-31 Thread Florin Coras
As mentioned previously, that is not supported for connects currently.  

Regards,
Florin

> On Mar 31, 2022, at 9:44 PM, weizhen9...@163.com wrote:
> 
> Hi,
> I describe our scene in detail. We use nginx in vpp host stack as a proxy. 
> And we add some features in nginx. For example, nginx close upstream tcp 
> links actively and this causes a lot of TIME_WAIT states in nginx proxy when 
> we test the performance of nginx using vpp host stack. So we configure kernel 
> parameters to multiplex tcp port and recycle port quickly. But I can't find 
> vpp host stack parameters which is similar with kernel parameters.
> How can we configure vpp parameters to reuse tcp port and recycle port 
> quickly?
> Thanks.  
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21177): https://lists.fd.io/g/vpp-dev/message/21177
Mute This Topic: https://lists.fd.io/mt/90157992/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: Private: Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

2022-03-31 Thread weizhen9612
Hi,
I describe our scene in detail. We use nginx in vpp host stack as a proxy. And 
we add some features in nginx. For example, nginx close upstream tcp links 
actively and this causes a lot of TIME_WAIT states in nginx proxy when we test 
the performance of nginx using vpp host stack. So we configure kernel 
parameters to multiplex tcp port and recycle port quickly. But I can't find vpp 
host stack parameters which is similar with kernel parameters.
How can we configure vpp parameters to reuse tcp port and recycle port quickly?
Thanks.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21176): https://lists.fd.io/g/vpp-dev/message/21176
Mute This Topic: https://lists.fd.io/mt/90157992/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: Private: Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

2022-03-31 Thread Florin Coras
Hi,

Given that 20k connections are being actively opened, those on main thread, and 
40k are established, those on the workers, suggests that tcp runs out of ports 
for connects. If possible either increase the number of destination IPs for 
nginx or try “tcp src-address ip1-ip2” and pass in a range of consecutive ips 
that could be used the vpp as source ips.   

The two workers seem to be pretty lightly loaded since loops/2 is over 2M, 
i.e., they’re mostly spinning empty, without work. 

Regards,
Florin

> On Mar 31, 2022, at 8:21 PM, weizhen9...@163.com wrote:
> 
> Hi,
> 
> From this test results, can you see some errors? Why is the performance of 
> nginx using vpp low?
> 
> Thanks. 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21175): https://lists.fd.io/g/vpp-dev/message/21175
Mute This Topic: https://lists.fd.io/mt/90157992/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: Private: Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

2022-03-31 Thread weizhen9612
Hi,

>From this test results, can you see some errors? Why is the performance of 
>nginx using vpp low?

Thanks.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21174): https://lists.fd.io/g/vpp-dev/message/21174
Mute This Topic: https://lists.fd.io/mt/90157992/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: Private: Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

2022-03-31 Thread Florin Coras
TCP accepts connections in time-wait (see here [1]) but won’t reuse ports of 
connection in time-wait for connects. If you expect lots of active opens from 
nginx to only one destination ip and you have multiple source ips, you could 
try to use "tcp src-address” cli. 

Regards,
Florin

[1] https://git.fd.io/vpp/tree/src/vnet/tcp/tcp_input.c#n2667

> On Mar 31, 2022, at 7:00 PM, weizhen9...@163.com wrote:
> 
> Hi, 
> 
> When we test the performance of nginx using vpp host stack, we execute the 
> following command.
> #show run 
> 
> 
> 
> 
> 
> #show session
> 
> 
> 
> Besides, we add some new features in nginx and this feature will cause a lot 
> of time_wait states. In order to reuse TIME_WAIT state tcp_tw_reuse and 
> tcp_tw_recycle. Besides, to make the port unrestricted, we modify other 
> parameters.
> So how can we modify the vpp host stack parameters?
> Thanks.
> 
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21173): https://lists.fd.io/g/vpp-dev/message/21173
Mute This Topic: https://lists.fd.io/mt/90157992/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: Private: Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

2022-03-31 Thread Florin Coras
I didn’t mean you should switch to envoy, just that throughput is pretty low 
probably because of some configuration. What that configuration is is not 
obvious unfortunately. 

Regarding the kernel parameters, we have time wait reuse enabled (equivalent to 
tcp_tw_reuse) but that should not matter unless nginx establishes a new 
connection for each request. Does it, as part of the features you’ve added?

Could you do: 
- clear run; show run - in vpp when under load
- show sessions - to see the number of session in vpp 

Also, could you reduce the number of workers to maybe 1-2 in nginx and 1 in 
vpp, to get some base performance readings?

Regards,
Florin 

[1] https://wiki.fd.io/images/0/08/Using_vpp_as_envoys_network_stack.pdf 


> On Mar 31, 2022, at 8:56 AM, weizhen9...@163.com wrote:
> 
> Hi,
> 
> I'm doing proxying with nginx. And we develop some new functions in nginx. 
> The performance of nginx in kernel host stack is higher than nginx using vpp 
> host stack. When testing the nginx in kernel host stack, we modify the kernel 
> parameters in kernel host stack. When using vpp host stack, how can we modify 
> the similar parameters?
> 
> Besides, we don't use the envoy. We develop some functions in nginx.
>> 
>> Hi, 
>> 
>> Spoke too soon. Noticed you’re doing proxying with nginx. 
>> 
>> What does clear run; show run report in vpp when under load?
>> 
>> Side note, with envoy I’m seeing much better numbers. See for instance slide 
>> 12 here [1]. So I suspect this is a configuration issue but can’t tell 
>> upfront what it is.
>> 
>> Regards,
>> Florin

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21168): https://lists.fd.io/g/vpp-dev/message/21168
Mute This Topic: https://lists.fd.io/mt/90157992/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

2022-03-31 Thread weizhen9612
Hi,
The version of vpp is 22.06. The configs are in the attachment.
The nginx using vpp host stack is low.

The most important test indicators is RPS.
How can we increase the performance of nginx using vpp host stack?


nginx.conf
Description: Binary data


startup.conf
Description: Binary data


vcl.conf
Description: Binary data

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21163): https://lists.fd.io/g/vpp-dev/message/21163
Mute This Topic: https://lists.fd.io/mt/90154397/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

2022-03-31 Thread Florin Coras
Hi, 

Could you provide a bit more info about the numbers you are seeing and the type 
of test you are doing? Also some details about your configs and vpp version? 

As for tcp_tw_recycle, I believe that was removed in Linux 4.12 because it was 
causing issues for nat-ed connections. Did you mean tcp_tw_reuse? If yes, are 
you testing CPS with only one remote client IP? 

Regards, 
Florin

> On Mar 31, 2022, at 6:35 AM, weizhen9...@163.com wrote:
> 
> Now, we develop some functions in nginx and test nginx performance in kernel 
> host stack and vpp host stack. We found the performance of nginx in vpp is 
> lower than nginx in kernel.
> So I want to ask how to debug to increase the performance of nginx in vpp. 
> Besides, we modify some parameter in kernel. For example, in file 
> /etc/sysctl.conf, we add the following parameters:
> net.ipv4.tcp_max_tw_buckets = 3000
> net.ipv4.tcp_tw_reuse = 1
> net.ipv4.tcp_tw_recycle = 1
> In file /etc/security/limited.conf, we add the following parameters:
> * soft nofile 102400
> * hard nofile 102400
> So how can we add those parameters in vpp host stack? And how can we increase 
> the performance of nginx in vpp?
> Thanks.
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21162): https://lists.fd.io/g/vpp-dev/message/21162
Mute This Topic: https://lists.fd.io/mt/90154397/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[vpp-dev] increase nginx performace using vpp host stack#nginx #vpp-hoststack

2022-03-31 Thread weizhen9612
Now, we develop some functions in nginx and test nginx performance in kernel 
host stack and vpp host stack. We found the performance of nginx in vpp is 
lower than nginx in kernel.
So I want to ask how to debug to increase the performance of nginx in vpp. 
Besides, we modify some parameter in kernel. For example, in file 
/etc/sysctl.conf, we add the following parameters:
net.ipv4.tcp_max_tw_buckets = 3000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
In file /etc/security/limited.conf, we add the following parameters:
* soft nofile 102400
* hard nofile 102400
So how can we add those parameters in vpp host stack? And how can we increase 
the performance of nginx in vpp?
Thanks.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21161): https://lists.fd.io/g/vpp-dev/message/21161
Mute This Topic: https://lists.fd.io/mt/90154397/21656
Mute #vpp-hoststack:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-hoststack
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-