Re: [ovs-discuss] Discussion on the logical rationality of flow-limit

2021-05-11 Thread taoyunupt
Hi, Ben
 I summit a patch just now, and CC to you.
 From this change ,  it adjust the algorithm to make flow-limit  increase 
faster as it decreases.  And I found that , the problem I mentioned in the past 
mails , also suit for Non-offload situation.
 The test result of Requests per second of Apache benchmark can reach 12k 
from 5k, the test command is as follows:


 ab -n 20 -c 500 -r http://1.1.1.2/




Thanks, 

Yun

在 2021-05-07 00:50:06,"Ben Pfaff"  写道:
>On Thu, May 06, 2021 at 09:58:57AM +0800, taoyunupt wrote:
>> 在 2021-05-06 03:26:46,"Ben Pfaff"  写道:
>> 
>> >On Fri, Apr 30, 2021 at 06:10:43PM +0800, taoyunupt wrote:
>> >> 
>> >> 
>> >> 
>> >> At 2021-04-29 06:39:11, "Ben Pfaff"  wrote:
>> >> >On Wed, Apr 28, 2021 at 08:12:06PM +0800, taoyunupt wrote:
>> >> >> Hi,
>> >> >>  Recently I encountered a TCP connection performance problem, the 
>> >> >> test tool is Apache benchmark.
>> >> >>  The OVS  in my environment is set for  hardware offload solution. 
>> >> >>  The "Requests per second" is about 6000/s, it closed to non-offload 
>> >> >> solution.
>> >> >> 
>> >> >> 
>> >> >>   "flow-lmit"  has a dynamic balance in udpif_revalidator, it will 
>> >> >> modify by the OVS condition(which is pind to "duration").   In the 
>> >> >> revalidate function, when the number of flows is greater than twice 
>> >> >> the "flow-limit" , the delete flow operation will be triggered to 
>> >> >> delete all flows; when the number of flows is greater than the 
>> >> >> "flow-limit", the aging time will be adjusted to 0.1s, Slowly delete 
>> >> >> flow.   
>> >> >> 
>> >> >> 
>> >> >>  
>> >> >>  I found that the reason for the poor performance is that when the 
>> >> >> number of flows in the datapath increases and the processing power of 
>> >> >> OVS decreases, a large number of flow deletions are generated. 
>> >> >>  As we know, In the hardware offloading scenario, although there 
>> >> >> are a lot of flows, in fact, apart from the first packet, there is no 
>> >> >> need to process subsequent packets. 
>> >> >>  In my opinion, the dynamic balance mechanism is very necessary, 
>> >> >> but we need to increase the value of “duration”, or provide some new 
>> >> >> switches for some high-performance scenarios, such as hardware 
>> >> >> offloading.
>> >> >>  Do we still need to restrict the number of flows so strictly? By 
>> >> >> the way, do you have another solution to resolve this?   
>> >> >
>> >> >It's been a long time since I worked on this, but I recall two reasons
>> >> >for the flow limit.  First, each flow takes up memory.  Second, each
>> >> >flow must be revalidated periodically, meaning that it uses CPU as
>> >> >well.
>> >> >
>> >> >I don't, off-hand, remember the real reasons why the logic for deleting
>> >> >flows works as it does.  It might be in the comments or the commit
>> >> >messages.  But, I suspect, it is because above the flow-limit we want to
>> >> >try to reduce the amount of memory and CPU time dedicated to the cache
>> >> >and, if we arrive at twice the flow limit, we conclude that that try
>> >> >failed and that we must have a large number of very short flows so that
>> >> >caching is not very valuable anyhow.
>> >> >
>> >> >In a hardware offload scenario, we get rid of some costs (the cost of
>> >> >processing and forwarding packets and perhaps the memory cost in the
>> >> >datapath) but we still have the cost of revalidating them.  When there
>> >> >are many flows, we add the extra cost of balancing flows between
>> >> >software and the offload hardware.
>> >> >
>> >> >Because of the remaining cost and the added ones when there is hardware
>> >> >offload, it's not obvious to me that we can stop limiting the number of
>> >> >flows.  I think that experimentation and measurements would be needed.
>> >> >Perhaps this would be an adjustment to the dynamic algorithm, rather
>> >> 
>> >> >than a removal of it.
>> >> 
>> >> 
>> >> I think we can increase the init `flow_limit` in udpif_create,1 is a 
>> >> small number for current server and OS, and if 'duration' is small ,we 
>> >> should increase faster by a lager number not `flow_limit += 1000;`.
>> >> I have not better idea for this situation. Do you have some suggestion? I 
>> >> am very glad to do this change.
>> >
>> >What kind of number are you thinking about?  I'd like to come up with a
>> >rationale for choosing it.  It might be even better to come up with an
>> 
>> >algorithm or a heuristic for choosing it.
>> 
>> 
>> I think we could set the initial value to 200,000, and adjust the
>> increase to 20,000 each time.  Can you describe the rationale
>> algorithm you meationed in detailed ?
>
>I'd expect that whoever is changing it would propose the rationale.
>
>I believe that part of the current rationale is to keep the limit at a
>level such that revalidation takes no more than 1 second.  That's an

>important aspect too.








Re: [ovs-discuss] Discussion on the logical rationality of flow-limit

2021-05-06 Thread taoyunupt






在 2021-05-07 00:50:06,"Ben Pfaff"  写道:
>On Thu, May 06, 2021 at 09:58:57AM +0800, taoyunupt wrote:
>> 在 2021-05-06 03:26:46,"Ben Pfaff"  写道:
>> 
>> >On Fri, Apr 30, 2021 at 06:10:43PM +0800, taoyunupt wrote:
>> >> 
>> >> 
>> >> 
>> >> At 2021-04-29 06:39:11, "Ben Pfaff"  wrote:
>> >> >On Wed, Apr 28, 2021 at 08:12:06PM +0800, taoyunupt wrote:
>> >> >> Hi,
>> >> >>  Recently I encountered a TCP connection performance problem, the 
>> >> >> test tool is Apache benchmark.
>> >> >>  The OVS  in my environment is set for  hardware offload solution. 
>> >> >>  The "Requests per second" is about 6000/s, it closed to non-offload 
>> >> >> solution.
>> >> >> 
>> >> >> 
>> >> >>   "flow-lmit"  has a dynamic balance in udpif_revalidator, it will 
>> >> >> modify by the OVS condition(which is pind to "duration").   In the 
>> >> >> revalidate function, when the number of flows is greater than twice 
>> >> >> the "flow-limit" , the delete flow operation will be triggered to 
>> >> >> delete all flows; when the number of flows is greater than the 
>> >> >> "flow-limit", the aging time will be adjusted to 0.1s, Slowly delete 
>> >> >> flow.   
>> >> >> 
>> >> >> 
>> >> >>  
>> >> >>  I found that the reason for the poor performance is that when the 
>> >> >> number of flows in the datapath increases and the processing power of 
>> >> >> OVS decreases, a large number of flow deletions are generated. 
>> >> >>  As we know, In the hardware offloading scenario, although there 
>> >> >> are a lot of flows, in fact, apart from the first packet, there is no 
>> >> >> need to process subsequent packets. 
>> >> >>  In my opinion, the dynamic balance mechanism is very necessary, 
>> >> >> but we need to increase the value of “duration”, or provide some new 
>> >> >> switches for some high-performance scenarios, such as hardware 
>> >> >> offloading.
>> >> >>  Do we still need to restrict the number of flows so strictly? By 
>> >> >> the way, do you have another solution to resolve this?   
>> >> >
>> >> >It's been a long time since I worked on this, but I recall two reasons
>> >> >for the flow limit.  First, each flow takes up memory.  Second, each
>> >> >flow must be revalidated periodically, meaning that it uses CPU as
>> >> >well.
>> >> >
>> >> >I don't, off-hand, remember the real reasons why the logic for deleting
>> >> >flows works as it does.  It might be in the comments or the commit
>> >> >messages.  But, I suspect, it is because above the flow-limit we want to
>> >> >try to reduce the amount of memory and CPU time dedicated to the cache
>> >> >and, if we arrive at twice the flow limit, we conclude that that try
>> >> >failed and that we must have a large number of very short flows so that
>> >> >caching is not very valuable anyhow.
>> >> >
>> >> >In a hardware offload scenario, we get rid of some costs (the cost of
>> >> >processing and forwarding packets and perhaps the memory cost in the
>> >> >datapath) but we still have the cost of revalidating them.  When there
>> >> >are many flows, we add the extra cost of balancing flows between
>> >> >software and the offload hardware.
>> >> >
>> >> >Because of the remaining cost and the added ones when there is hardware
>> >> >offload, it's not obvious to me that we can stop limiting the number of
>> >> >flows.  I think that experimentation and measurements would be needed.
>> >> >Perhaps this would be an adjustment to the dynamic algorithm, rather
>> >> 
>> >> >than a removal of it.
>> >> 
>> >> 
>> >> I think we can increase the init `flow_limit` in udpif_create,1 is a 
>> >> small number for current server and OS, and if 'duration' is small ,we 
>> >> should increase faster by a lager number not `flow_limit += 1000;`.
>> >> I have not better idea for this situation. Do you have some suggestion? I 
>> >> am very glad to do this change.
>> >
>> >What kind of number are you thinking about?  I'd like to come up with a
>> >rationale for choosing it.  It might be even better to come up with an
>> 
>> >algorithm or a heuristic for choosing it.
>> 
>> 
>> I think we could set the initial value to 200,000, and adjust the
>> increase to 20,000 each time.  Can you describe the rationale
>> algorithm you meationed in detailed ?
>
>I'd expect that whoever is changing it would propose the rationale.
>
>I believe that part of the current rationale is to keep the limit at a
>level such that revalidation takes no more than 1 second.  That's an

>important aspect too.


I agree with you about revalidation needs take no more than 1 second.
But, in case of the number of flows is 65535, and the max_revalidator_default 
is 0.5s, we need  about 32s ((65535-1000/1000)*0.5s) to get to the 
number(65535). If the number get larger, it will cost more time .


The difference between offloaded and non-offloaded scenarios is whether OVS 
needs to process subsequent packages. In the offloaded scenario, even if the 
number of flows is large, it will not bring 

Re: [ovs-discuss] Discussion on the logical rationality of flow-limit

2021-05-06 Thread Ben Pfaff
On Thu, May 06, 2021 at 09:58:57AM +0800, taoyunupt wrote:
> 在 2021-05-06 03:26:46,"Ben Pfaff"  写道:
> 
> >On Fri, Apr 30, 2021 at 06:10:43PM +0800, taoyunupt wrote:
> >> 
> >> 
> >> 
> >> At 2021-04-29 06:39:11, "Ben Pfaff"  wrote:
> >> >On Wed, Apr 28, 2021 at 08:12:06PM +0800, taoyunupt wrote:
> >> >> Hi,
> >> >>  Recently I encountered a TCP connection performance problem, the 
> >> >> test tool is Apache benchmark.
> >> >>  The OVS  in my environment is set for  hardware offload solution.  
> >> >> The "Requests per second" is about 6000/s, it closed to non-offload 
> >> >> solution.
> >> >> 
> >> >> 
> >> >>   "flow-lmit"  has a dynamic balance in udpif_revalidator, it will 
> >> >> modify by the OVS condition(which is pind to "duration").   In the 
> >> >> revalidate function, when the number of flows is greater than twice the 
> >> >> "flow-limit" , the delete flow operation will be triggered to delete 
> >> >> all flows; when the number of flows is greater than the "flow-limit", 
> >> >> the aging time will be adjusted to 0.1s, Slowly delete flow.   
> >> >> 
> >> >> 
> >> >>  
> >> >>  I found that the reason for the poor performance is that when the 
> >> >> number of flows in the datapath increases and the processing power of 
> >> >> OVS decreases, a large number of flow deletions are generated. 
> >> >>  As we know, In the hardware offloading scenario, although there 
> >> >> are a lot of flows, in fact, apart from the first packet, there is no 
> >> >> need to process subsequent packets. 
> >> >>  In my opinion, the dynamic balance mechanism is very necessary, 
> >> >> but we need to increase the value of “duration”, or provide some new 
> >> >> switches for some high-performance scenarios, such as hardware 
> >> >> offloading.
> >> >>  Do we still need to restrict the number of flows so strictly? By 
> >> >> the way, do you have another solution to resolve this?   
> >> >
> >> >It's been a long time since I worked on this, but I recall two reasons
> >> >for the flow limit.  First, each flow takes up memory.  Second, each
> >> >flow must be revalidated periodically, meaning that it uses CPU as
> >> >well.
> >> >
> >> >I don't, off-hand, remember the real reasons why the logic for deleting
> >> >flows works as it does.  It might be in the comments or the commit
> >> >messages.  But, I suspect, it is because above the flow-limit we want to
> >> >try to reduce the amount of memory and CPU time dedicated to the cache
> >> >and, if we arrive at twice the flow limit, we conclude that that try
> >> >failed and that we must have a large number of very short flows so that
> >> >caching is not very valuable anyhow.
> >> >
> >> >In a hardware offload scenario, we get rid of some costs (the cost of
> >> >processing and forwarding packets and perhaps the memory cost in the
> >> >datapath) but we still have the cost of revalidating them.  When there
> >> >are many flows, we add the extra cost of balancing flows between
> >> >software and the offload hardware.
> >> >
> >> >Because of the remaining cost and the added ones when there is hardware
> >> >offload, it's not obvious to me that we can stop limiting the number of
> >> >flows.  I think that experimentation and measurements would be needed.
> >> >Perhaps this would be an adjustment to the dynamic algorithm, rather
> >> 
> >> >than a removal of it.
> >> 
> >> 
> >> I think we can increase the init `flow_limit` in udpif_create,1 is a 
> >> small number for current server and OS, and if 'duration' is small ,we 
> >> should increase faster by a lager number not `flow_limit += 1000;`.
> >> I have not better idea for this situation. Do you have some suggestion? I 
> >> am very glad to do this change.
> >
> >What kind of number are you thinking about?  I'd like to come up with a
> >rationale for choosing it.  It might be even better to come up with an
> 
> >algorithm or a heuristic for choosing it.
> 
> 
> I think we could set the initial value to 200,000, and adjust the
> increase to 20,000 each time.  Can you describe the rationale
> algorithm you meationed in detailed ?

I'd expect that whoever is changing it would propose the rationale.

I believe that part of the current rationale is to keep the limit at a
level such that revalidation takes no more than 1 second.  That's an
important aspect too.
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss


Re: [ovs-discuss] Discussion on the logical rationality of flow-limit

2021-05-05 Thread taoyunupt
在 2021-05-06 03:26:46,"Ben Pfaff"  写道:

>On Fri, Apr 30, 2021 at 06:10:43PM +0800, taoyunupt wrote:
>> 
>> 
>> 
>> At 2021-04-29 06:39:11, "Ben Pfaff"  wrote:
>> >On Wed, Apr 28, 2021 at 08:12:06PM +0800, taoyunupt wrote:
>> >> Hi,
>> >>  Recently I encountered a TCP connection performance problem, the 
>> >> test tool is Apache benchmark.
>> >>  The OVS  in my environment is set for  hardware offload solution.  
>> >> The "Requests per second" is about 6000/s, it closed to non-offload 
>> >> solution.
>> >> 
>> >> 
>> >>   "flow-lmit"  has a dynamic balance in udpif_revalidator, it will 
>> >> modify by the OVS condition(which is pind to "duration").   In the 
>> >> revalidate function, when the number of flows is greater than twice the 
>> >> "flow-limit" , the delete flow operation will be triggered to delete all 
>> >> flows; when the number of flows is greater than the "flow-limit", the 
>> >> aging time will be adjusted to 0.1s, Slowly delete flow.   
>> >> 
>> >> 
>> >>  
>> >>  I found that the reason for the poor performance is that when the 
>> >> number of flows in the datapath increases and the processing power of OVS 
>> >> decreases, a large number of flow deletions are generated. 
>> >>  As we know, In the hardware offloading scenario, although there are 
>> >> a lot of flows, in fact, apart from the first packet, there is no need to 
>> >> process subsequent packets. 
>> >>  In my opinion, the dynamic balance mechanism is very necessary, but 
>> >> we need to increase the value of “duration”, or provide some new switches 
>> >> for some high-performance scenarios, such as hardware offloading.
>> >>  Do we still need to restrict the number of flows so strictly? By the 
>> >> way, do you have another solution to resolve this?   
>> >
>> >It's been a long time since I worked on this, but I recall two reasons
>> >for the flow limit.  First, each flow takes up memory.  Second, each
>> >flow must be revalidated periodically, meaning that it uses CPU as
>> >well.
>> >
>> >I don't, off-hand, remember the real reasons why the logic for deleting
>> >flows works as it does.  It might be in the comments or the commit
>> >messages.  But, I suspect, it is because above the flow-limit we want to
>> >try to reduce the amount of memory and CPU time dedicated to the cache
>> >and, if we arrive at twice the flow limit, we conclude that that try
>> >failed and that we must have a large number of very short flows so that
>> >caching is not very valuable anyhow.
>> >
>> >In a hardware offload scenario, we get rid of some costs (the cost of
>> >processing and forwarding packets and perhaps the memory cost in the
>> >datapath) but we still have the cost of revalidating them.  When there
>> >are many flows, we add the extra cost of balancing flows between
>> >software and the offload hardware.
>> >
>> >Because of the remaining cost and the added ones when there is hardware
>> >offload, it's not obvious to me that we can stop limiting the number of
>> >flows.  I think that experimentation and measurements would be needed.
>> >Perhaps this would be an adjustment to the dynamic algorithm, rather
>> 
>> >than a removal of it.
>> 
>> 
>> I think we can increase the init `flow_limit` in udpif_create,1 is a 
>> small number for current server and OS, and if 'duration' is small ,we 
>> should increase faster by a lager number not `flow_limit += 1000;`.
>> I have not better idea for this situation. Do you have some suggestion? I am 
>> very glad to do this change.
>
>What kind of number are you thinking about?  I'd like to come up with a
>rationale for choosing it.  It might be even better to come up with an

>algorithm or a heuristic for choosing it.


I think we could  set the initial value to 200,000, and adjust the increase to 
20,000 each time.  Can  you describe the rationale algorithm you meationed in 
detailed ? ___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss


Re: [ovs-discuss] Discussion on the logical rationality of flow-limit

2021-05-05 Thread Ben Pfaff
On Fri, Apr 30, 2021 at 06:10:43PM +0800, taoyunupt wrote:
> 
> 
> 
> At 2021-04-29 06:39:11, "Ben Pfaff"  wrote:
> >On Wed, Apr 28, 2021 at 08:12:06PM +0800, taoyunupt wrote:
> >> Hi,
> >>  Recently I encountered a TCP connection performance problem, the test 
> >> tool is Apache benchmark.
> >>  The OVS  in my environment is set for  hardware offload solution.  
> >> The "Requests per second" is about 6000/s, it closed to non-offload 
> >> solution.
> >> 
> >> 
> >>   "flow-lmit"  has a dynamic balance in udpif_revalidator, it will 
> >> modify by the OVS condition(which is pind to "duration").   In the 
> >> revalidate function, when the number of flows is greater than twice the 
> >> "flow-limit" , the delete flow operation will be triggered to delete all 
> >> flows; when the number of flows is greater than the "flow-limit", the 
> >> aging time will be adjusted to 0.1s, Slowly delete flow.   
> >> 
> >> 
> >>  
> >>  I found that the reason for the poor performance is that when the 
> >> number of flows in the datapath increases and the processing power of OVS 
> >> decreases, a large number of flow deletions are generated. 
> >>  As we know, In the hardware offloading scenario, although there are a 
> >> lot of flows, in fact, apart from the first packet, there is no need to 
> >> process subsequent packets. 
> >>  In my opinion, the dynamic balance mechanism is very necessary, but 
> >> we need to increase the value of “duration”, or provide some new switches 
> >> for some high-performance scenarios, such as hardware offloading.
> >>  Do we still need to restrict the number of flows so strictly? By the 
> >> way, do you have another solution to resolve this?   
> >
> >It's been a long time since I worked on this, but I recall two reasons
> >for the flow limit.  First, each flow takes up memory.  Second, each
> >flow must be revalidated periodically, meaning that it uses CPU as
> >well.
> >
> >I don't, off-hand, remember the real reasons why the logic for deleting
> >flows works as it does.  It might be in the comments or the commit
> >messages.  But, I suspect, it is because above the flow-limit we want to
> >try to reduce the amount of memory and CPU time dedicated to the cache
> >and, if we arrive at twice the flow limit, we conclude that that try
> >failed and that we must have a large number of very short flows so that
> >caching is not very valuable anyhow.
> >
> >In a hardware offload scenario, we get rid of some costs (the cost of
> >processing and forwarding packets and perhaps the memory cost in the
> >datapath) but we still have the cost of revalidating them.  When there
> >are many flows, we add the extra cost of balancing flows between
> >software and the offload hardware.
> >
> >Because of the remaining cost and the added ones when there is hardware
> >offload, it's not obvious to me that we can stop limiting the number of
> >flows.  I think that experimentation and measurements would be needed.
> >Perhaps this would be an adjustment to the dynamic algorithm, rather
> 
> >than a removal of it.
> 
> 
> I think we can increase the init `flow_limit` in udpif_create,1 is a 
> small number for current server and OS, and if 'duration' is small ,we should 
> increase faster by a lager number not `flow_limit += 1000;`.
> I have not better idea for this situation. Do you have some suggestion? I am 
> very glad to do this change.

What kind of number are you thinking about?  I'd like to come up with a
rationale for choosing it.  It might be even better to come up with an
algorithm or a heuristic for choosing it.
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss


Re: [ovs-discuss] Discussion on the logical rationality of flow-limit

2021-04-30 Thread taoyunupt



At 2021-04-29 06:39:11, "Ben Pfaff"  wrote:
>On Wed, Apr 28, 2021 at 08:12:06PM +0800, taoyunupt wrote:
>> Hi,
>>  Recently I encountered a TCP connection performance problem, the test 
>> tool is Apache benchmark.
>>  The OVS  in my environment is set for  hardware offload solution.  The 
>> "Requests per second" is about 6000/s, it closed to non-offload solution.
>> 
>> 
>>   "flow-lmit"  has a dynamic balance in udpif_revalidator, it will 
>> modify by the OVS condition(which is pind to "duration").   In the 
>> revalidate function, when the number of flows is greater than twice the 
>> "flow-limit" , the delete flow operation will be triggered to delete all 
>> flows; when the number of flows is greater than the "flow-limit", the aging 
>> time will be adjusted to 0.1s, Slowly delete flow.   
>> 
>> 
>>  
>>  I found that the reason for the poor performance is that when the 
>> number of flows in the datapath increases and the processing power of OVS 
>> decreases, a large number of flow deletions are generated. 
>>  As we know, In the hardware offloading scenario, although there are a 
>> lot of flows, in fact, apart from the first packet, there is no need to 
>> process subsequent packets. 
>>  In my opinion, the dynamic balance mechanism is very necessary, but we 
>> need to increase the value of “duration”, or provide some new switches for 
>> some high-performance scenarios, such as hardware offloading.
>>  Do we still need to restrict the number of flows so strictly? By the 
>> way, do you have another solution to resolve this?   
>
>It's been a long time since I worked on this, but I recall two reasons
>for the flow limit.  First, each flow takes up memory.  Second, each
>flow must be revalidated periodically, meaning that it uses CPU as
>well.
>
>I don't, off-hand, remember the real reasons why the logic for deleting
>flows works as it does.  It might be in the comments or the commit
>messages.  But, I suspect, it is because above the flow-limit we want to
>try to reduce the amount of memory and CPU time dedicated to the cache
>and, if we arrive at twice the flow limit, we conclude that that try
>failed and that we must have a large number of very short flows so that
>caching is not very valuable anyhow.
>
>In a hardware offload scenario, we get rid of some costs (the cost of
>processing and forwarding packets and perhaps the memory cost in the
>datapath) but we still have the cost of revalidating them.  When there
>are many flows, we add the extra cost of balancing flows between
>software and the offload hardware.
>
>Because of the remaining cost and the added ones when there is hardware
>offload, it's not obvious to me that we can stop limiting the number of
>flows.  I think that experimentation and measurements would be needed.
>Perhaps this would be an adjustment to the dynamic algorithm, rather

>than a removal of it.


I think we can increase the init `flow_limit` in udpif_create,1 is a small 
number for current server and OS, and if 'duration' is small ,we should 
increase faster by a lager number not `flow_limit += 1000;`.
I have not better idea for this situation. Do you have some suggestion? I am 
very glad to do this change.





___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss


Re: [ovs-discuss] Discussion on the logical rationality of flow-limit

2021-04-28 Thread Ben Pfaff
On Wed, Apr 28, 2021 at 08:12:06PM +0800, taoyunupt wrote:
> Hi,
>  Recently I encountered a TCP connection performance problem, the test 
> tool is Apache benchmark.
>  The OVS  in my environment is set for  hardware offload solution.  The 
> "Requests per second" is about 6000/s, it closed to non-offload solution.
> 
> 
>   "flow-lmit"  has a dynamic balance in udpif_revalidator, it will modify 
> by the OVS condition(which is pind to "duration").   In the revalidate 
> function, when the number of flows is greater than twice the "flow-limit" , 
> the delete flow operation will be triggered to delete all flows; when the 
> number of flows is greater than the "flow-limit", the aging time will be 
> adjusted to 0.1s, Slowly delete flow.   
> 
> 
>  
>  I found that the reason for the poor performance is that when the number 
> of flows in the datapath increases and the processing power of OVS decreases, 
> a large number of flow deletions are generated. 
>  As we know, In the hardware offloading scenario, although there are a 
> lot of flows, in fact, apart from the first packet, there is no need to 
> process subsequent packets. 
>  In my opinion, the dynamic balance mechanism is very necessary, but we 
> need to increase the value of “duration”, or provide some new switches for 
> some high-performance scenarios, such as hardware offloading.
>  Do we still need to restrict the number of flows so strictly? By the 
> way, do you have another solution to resolve this?   

It's been a long time since I worked on this, but I recall two reasons
for the flow limit.  First, each flow takes up memory.  Second, each
flow must be revalidated periodically, meaning that it uses CPU as
well.

I don't, off-hand, remember the real reasons why the logic for deleting
flows works as it does.  It might be in the comments or the commit
messages.  But, I suspect, it is because above the flow-limit we want to
try to reduce the amount of memory and CPU time dedicated to the cache
and, if we arrive at twice the flow limit, we conclude that that try
failed and that we must have a large number of very short flows so that
caching is not very valuable anyhow.

In a hardware offload scenario, we get rid of some costs (the cost of
processing and forwarding packets and perhaps the memory cost in the
datapath) but we still have the cost of revalidating them.  When there
are many flows, we add the extra cost of balancing flows between
software and the offload hardware.

Because of the remaining cost and the added ones when there is hardware
offload, it's not obvious to me that we can stop limiting the number of
flows.  I think that experimentation and measurements would be needed.
Perhaps this would be an adjustment to the dynamic algorithm, rather
than a removal of it.
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss


[ovs-discuss] Discussion on the logical rationality of flow-limit

2021-04-28 Thread taoyunupt
Hi,
 Recently I encountered a TCP connection performance problem, the test tool 
is Apache benchmark.
 The OVS  in my environment is set for  hardware offload solution.  The 
"Requests per second" is about 6000/s, it closed to non-offload solution.


  "flow-lmit"  has a dynamic balance in udpif_revalidator, it will modify 
by the OVS condition(which is pind to "duration").   In the revalidate 
function, when the number of flows is greater than twice the "flow-limit" , the 
delete flow operation will be triggered to delete all flows; when the number of 
flows is greater than the "flow-limit", the aging time will be adjusted to 
0.1s, Slowly delete flow.   


 
 I found that the reason for the poor performance is that when the number 
of flows in the datapath increases and the processing power of OVS decreases, a 
large number of flow deletions are generated. 
 As we know, In the hardware offloading scenario, although there are a lot 
of flows, in fact, apart from the first packet, there is no need to process 
subsequent packets. 
 In my opinion, the dynamic balance mechanism is very necessary, but we 
need to increase the value of “duration”, or provide some new switches for some 
high-performance scenarios, such as hardware offloading.
 Do we still need to restrict the number of flows so strictly? By the way, 
do you have another solution to resolve this?   




Thanks,
Yun___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss