Re: [ovs-dev] thundering herd wakeup of handler threads

2020-03-12 Thread Aaron Conole
David Ahern writes: > On 1/8/20 9:18 AM, Aaron Conole wrote: >> David Ahern writes: >> >>> On 12/16/19 2:42 PM, Aaron Conole wrote: Can you try the following and see if your scalability issue is addressed? I think it could be better integrated, but this is a different quick 'n

Re: [ovs-dev] thundering herd wakeup of handler threads

2020-03-11 Thread David Ahern
On 1/8/20 9:18 AM, Aaron Conole wrote: > David Ahern writes: > >> On 12/16/19 2:42 PM, Aaron Conole wrote: >>> Can you try the following and see if your scalability issue is >>> addressed? I think it could be better integrated, but this is a >>> different quick 'n dirty. >> >> your patch

Re: [ovs-dev] thundering herd wakeup of handler threads

2020-01-08 Thread Aaron Conole
David Ahern writes: > On 12/16/19 2:42 PM, Aaron Conole wrote: >> Can you try the following and see if your scalability issue is >> addressed? I think it could be better integrated, but this is a >> different quick 'n dirty. > > your patch reduces the number of threads awakened, but it is still

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-20 Thread David Ahern
On 12/16/19 2:42 PM, Aaron Conole wrote: > Can you try the following and see if your scalability issue is > addressed? I think it could be better integrated, but this is a > different quick 'n dirty. your patch reduces the number of threads awakened, but it is still really high - 43 out of 71

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-16 Thread David Ahern
On 12/16/19 2:42 PM, Aaron Conole wrote: > Can you try the following and see if your scalability issue is > addressed? I think it could be better integrated, but this is a > different quick 'n dirty. I'll to get to it before the end of the week. ___

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-16 Thread Aaron Conole
David Ahern writes: > On 12/13/19 1:52 PM, Aaron Conole wrote: >> Jason Baron via dev writes: >> >>> On 12/10/19 5:20 PM, David Ahern wrote: On 12/10/19 3:09 PM, Jason Baron wrote: > Hi David, > > The idea is that we try and queue new work to 'idle' threads in an > attempt

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-13 Thread David Ahern
On 12/13/19 1:52 PM, Aaron Conole wrote: > Jason Baron via dev writes: > >> On 12/10/19 5:20 PM, David Ahern wrote: >>> On 12/10/19 3:09 PM, Jason Baron wrote: Hi David, The idea is that we try and queue new work to 'idle' threads in an attempt to distribute a workload. Thus,

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-13 Thread Aaron Conole
Jason Baron via dev writes: > On 12/10/19 5:20 PM, David Ahern wrote: >> On 12/10/19 3:09 PM, Jason Baron wrote: >>> Hi David, >>> >>> The idea is that we try and queue new work to 'idle' threads in an >>> attempt to distribute a workload. Thus, once we find an 'idle' thread we >>> stop waking

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-10 Thread Jason Baron via dev
On 12/10/19 5:20 PM, David Ahern wrote: > On 12/10/19 3:09 PM, Jason Baron wrote: >> Hi David, >> >> The idea is that we try and queue new work to 'idle' threads in an >> attempt to distribute a workload. Thus, once we find an 'idle' thread we >> stop waking up other threads. While we are

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-10 Thread Jason Baron via dev
On 12/10/19 4:00 PM, David Ahern wrote: > [ adding Jason as author of the patch that added the epoll exclusive flag ] > > On 12/10/19 12:37 PM, Matteo Croce wrote: >> On Tue, Dec 10, 2019 at 8:13 PM David Ahern wrote: >>> >>> Hi Matteo: >>> >>> On a hypervisor running a 4.14.91 kernel and OVS

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-10 Thread David Ahern
On 12/10/19 3:09 PM, Jason Baron wrote: > Hi David, > > The idea is that we try and queue new work to 'idle' threads in an > attempt to distribute a workload. Thus, once we find an 'idle' thread we > stop waking up other threads. While we are searching the wakeup list for > idle threads, we do

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-10 Thread David Ahern
On 12/10/19 2:20 PM, Matteo Croce wrote: > > Before this patch (which unfortunately is needed to avoid -EMFILE > errors with many ports), how many sockets are awakened when an ARP is > received? > on systems using 2.7.3 I see only a single handler thread awakened on upcalls. Yes, I saw the

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-10 Thread Matteo Croce
On Tue, Dec 10, 2019 at 10:00 PM David Ahern wrote: > > [ adding Jason as author of the patch that added the epoll exclusive flag ] > > On 12/10/19 12:37 PM, Matteo Croce wrote: > > On Tue, Dec 10, 2019 at 8:13 PM David Ahern wrote: > >> > >> Hi Matteo: > >> > >> On a hypervisor running a

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-10 Thread David Ahern
[ adding Jason as author of the patch that added the epoll exclusive flag ] On 12/10/19 12:37 PM, Matteo Croce wrote: > On Tue, Dec 10, 2019 at 8:13 PM David Ahern wrote: >> >> Hi Matteo: >> >> On a hypervisor running a 4.14.91 kernel and OVS 2.11 I am seeing a >> thundering herd wake up

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-10 Thread Matteo Croce
On Tue, Dec 10, 2019 at 8:41 PM David Ahern wrote: > > On 12/10/19 12:37 PM, Matteo Croce wrote: > > On Tue, Dec 10, 2019 at 8:13 PM David Ahern wrote: > >> > >> Hi Matteo: > >> > >> On a hypervisor running a 4.14.91 kernel and OVS 2.11 I am seeing a > >> thundering herd wake up problem. Every

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-10 Thread David Ahern
On 12/10/19 12:37 PM, Matteo Croce wrote: > On Tue, Dec 10, 2019 at 8:13 PM David Ahern wrote: >> >> Hi Matteo: >> >> On a hypervisor running a 4.14.91 kernel and OVS 2.11 I am seeing a >> thundering herd wake up problem. Every packet punted to userspace wakes >> up every one of the handler

Re: [ovs-dev] thundering herd wakeup of handler threads

2019-12-10 Thread Matteo Croce
On Tue, Dec 10, 2019 at 8:13 PM David Ahern wrote: > > Hi Matteo: > > On a hypervisor running a 4.14.91 kernel and OVS 2.11 I am seeing a > thundering herd wake up problem. Every packet punted to userspace wakes > up every one of the handler threads. On a box with 96 cpus, there are 71 > handler

[ovs-dev] thundering herd wakeup of handler threads

2019-12-10 Thread David Ahern
Hi Matteo: On a hypervisor running a 4.14.91 kernel and OVS 2.11 I am seeing a thundering herd wake up problem. Every packet punted to userspace wakes up every one of the handler threads. On a box with 96 cpus, there are 71 handler threads which means 71 process wakeups for every packet punted.