Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, May 30, 2017 at 01:17:41PM -0500, Christoph Lameter wrote: > On Fri, 26 May 2017, Marcelo Tosatti wrote: > > > > interrupts and scheduler ticks. But what does this have to do with vmstat? > > > > > > Show me your dpdk code running and trace the tick on / off events as well > > > as the

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, May 30, 2017 at 01:17:41PM -0500, Christoph Lameter wrote: > On Fri, 26 May 2017, Marcelo Tosatti wrote: > > > > interrupts and scheduler ticks. But what does this have to do with vmstat? > > > > > > Show me your dpdk code running and trace the tick on / off events as well > > > as the

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 26 May 2017, Marcelo Tosatti wrote: > > interrupts and scheduler ticks. But what does this have to do with vmstat? > > > > Show me your dpdk code running and trace the tick on / off events as well > > as the vmstat invocations. Also show all system calls occurring on the cpu > > that

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 26 May 2017, Marcelo Tosatti wrote: > > interrupts and scheduler ticks. But what does this have to do with vmstat? > > > > Show me your dpdk code running and trace the tick on / off events as well > > as the vmstat invocations. Also show all system calls occurring on the cpu > > that

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Thu, May 25, 2017 at 10:24:46PM -0500, Christoph Lameter wrote: > On Thu, 25 May 2017, Marcelo Tosatti wrote: > > > Argument? We're showing you the data that this is causing a latency > > problem for us. > > Sorry I am not sure where the data shows a latency problem. There are > interrupts

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Thu, May 25, 2017 at 10:24:46PM -0500, Christoph Lameter wrote: > On Thu, 25 May 2017, Marcelo Tosatti wrote: > > > Argument? We're showing you the data that this is causing a latency > > problem for us. > > Sorry I am not sure where the data shows a latency problem. There are > interrupts

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Thu, 25 May 2017, Marcelo Tosatti wrote: > Argument? We're showing you the data that this is causing a latency > problem for us. Sorry I am not sure where the data shows a latency problem. There are interrupts and scheduler ticks. But what does this have to do with vmstat? Show me your dpdk

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Thu, 25 May 2017, Marcelo Tosatti wrote: > Argument? We're showing you the data that this is causing a latency > problem for us. Sorry I am not sure where the data shows a latency problem. There are interrupts and scheduler ticks. But what does this have to do with vmstat? Show me your dpdk

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, May 19, 2017 at 01:49:34PM -0400, Luiz Capitulino wrote: > On Fri, 19 May 2017 12:13:26 -0500 (CDT) > Christoph Lameter wrote: > > > > So why are you against integrating this simple, isolated patch which > > > does not affect how current logic works? > > > > Frankly

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, May 19, 2017 at 01:49:34PM -0400, Luiz Capitulino wrote: > On Fri, 19 May 2017 12:13:26 -0500 (CDT) > Christoph Lameter wrote: > > > > So why are you against integrating this simple, isolated patch which > > > does not affect how current logic works? > > > > Frankly the argument does

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Mon, May 22, 2017 at 11:38:02AM -0500, Christoph Lameter wrote: > On Sat, 20 May 2017, Marcelo Tosatti wrote: > > > > And you can configure the interval of vmstat updates freely Set > > > the vmstat_interval to 60 seconds instead of 2 for a try? Is that rare > > > enough? > > > > Not rare

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Mon, May 22, 2017 at 11:38:02AM -0500, Christoph Lameter wrote: > On Sat, 20 May 2017, Marcelo Tosatti wrote: > > > > And you can configure the interval of vmstat updates freely Set > > > the vmstat_interval to 60 seconds instead of 2 for a try? Is that rare > > > enough? > > > > Not rare

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Sat, 20 May 2017, Marcelo Tosatti wrote: > > And you can configure the interval of vmstat updates freely Set > > the vmstat_interval to 60 seconds instead of 2 for a try? Is that rare > > enough? > > Not rare enough. Never is rare enough. Ok what about the other stuff that must be going

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Sat, 20 May 2017, Marcelo Tosatti wrote: > > And you can configure the interval of vmstat updates freely Set > > the vmstat_interval to 60 seconds instead of 2 for a try? Is that rare > > enough? > > Not rare enough. Never is rare enough. Ok what about the other stuff that must be going

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 19 May 2017, Luiz Capitulino wrote: > Something that crossed my mind was to add a new tunable to set > the vmstat_interval for each CPU, this way we could essentially > disable it to the CPUs where DPDK is running. What's the implications > of doing this besides not getting up to date

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 19 May 2017, Luiz Capitulino wrote: > Something that crossed my mind was to add a new tunable to set > the vmstat_interval for each CPU, this way we could essentially > disable it to the CPUs where DPDK is running. What's the implications > of doing this besides not getting up to date

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, May 19, 2017 at 12:13:26PM -0500, Christoph Lameter wrote: > On Fri, 19 May 2017, Marcelo Tosatti wrote: > > > Use-case: realtime application on an isolated core which for some reason > > updates vmstatistics. > > Ok that is already only happening every 2 seconds by default and that >

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, May 19, 2017 at 12:13:26PM -0500, Christoph Lameter wrote: > On Fri, 19 May 2017, Marcelo Tosatti wrote: > > > Use-case: realtime application on an isolated core which for some reason > > updates vmstatistics. > > Ok that is already only happening every 2 seconds by default and that >

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 19 May 2017 12:13:26 -0500 (CDT) Christoph Lameter wrote: > > So why are you against integrating this simple, isolated patch which > > does not affect how current logic works? > > Frankly the argument does not make sense. Vmstat updates occur very > infrequently

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 19 May 2017 12:13:26 -0500 (CDT) Christoph Lameter wrote: > > So why are you against integrating this simple, isolated patch which > > does not affect how current logic works? > > Frankly the argument does not make sense. Vmstat updates occur very > infrequently (probably even less

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 19 May 2017, Marcelo Tosatti wrote: > Use-case: realtime application on an isolated core which for some reason > updates vmstatistics. Ok that is already only happening every 2 seconds by default and that interval is configurable via the vmstat_interval proc setting. > > Just a

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 19 May 2017, Marcelo Tosatti wrote: > Use-case: realtime application on an isolated core which for some reason > updates vmstatistics. Ok that is already only happening every 2 seconds by default and that interval is configurable via the vmstat_interval proc setting. > > Just a

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

Hi Christoph, On Tue, May 16, 2017 at 08:37:11AM -0500, Christoph Lameter wrote: > On Mon, 15 May 2017, Marcelo Tosatti wrote: > > > > NOHZ already does that. I wanted to know what your problem is that you > > > see. The latency issue has already been solved as far as I can tell . > > > Please

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

Hi Christoph, On Tue, May 16, 2017 at 08:37:11AM -0500, Christoph Lameter wrote: > On Mon, 15 May 2017, Marcelo Tosatti wrote: > > > > NOHZ already does that. I wanted to know what your problem is that you > > > see. The latency issue has already been solved as far as I can tell . > > > Please

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Mon, 15 May 2017, Marcelo Tosatti wrote: > > NOHZ already does that. I wanted to know what your problem is that you > > see. The latency issue has already been solved as far as I can tell . > > Please tell me why the existing solutions are not sufficient for you. > > We don't want

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Mon, 15 May 2017, Marcelo Tosatti wrote: > > NOHZ already does that. I wanted to know what your problem is that you > > see. The latency issue has already been solved as far as I can tell . > > Please tell me why the existing solutions are not sufficient for you. > > We don't want

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, May 12, 2017 at 11:57:15AM -0500, Christoph Lameter wrote: > On Fri, 12 May 2017, Marcelo Tosatti wrote: > > > > What exactly is the issue you are seeing and want to address? I think we > > > have similar aims and as far as I know the current situation is already > > > good enough for

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, May 12, 2017 at 11:57:15AM -0500, Christoph Lameter wrote: > On Fri, 12 May 2017, Marcelo Tosatti wrote: > > > > What exactly is the issue you are seeing and want to address? I think we > > > have similar aims and as far as I know the current situation is already > > > good enough for

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 12 May 2017, Marcelo Tosatti wrote: > > What exactly is the issue you are seeing and want to address? I think we > > have similar aims and as far as I know the current situation is already > > good enough for what you may need. You may just not be aware of how to > > configure this. > > I

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 12 May 2017, Marcelo Tosatti wrote: > > What exactly is the issue you are seeing and want to address? I think we > > have similar aims and as far as I know the current situation is already > > good enough for what you may need. You may just not be aware of how to > > configure this. > > I

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, May 12, 2017 at 11:07:48AM -0500, Christoph Lameter wrote: > On Fri, 12 May 2017, Marcelo Tosatti wrote: > > > In our case, vmstat updates are very rare (CPU is dominated by DPDK). > > What is the OS doing on the cores that DPDK runs on? I mean we here can > clean a processor of all

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, May 12, 2017 at 11:07:48AM -0500, Christoph Lameter wrote: > On Fri, 12 May 2017, Marcelo Tosatti wrote: > > > In our case, vmstat updates are very rare (CPU is dominated by DPDK). > > What is the OS doing on the cores that DPDK runs on? I mean we here can > clean a processor of all

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 12 May 2017, Marcelo Tosatti wrote: > In our case, vmstat updates are very rare (CPU is dominated by DPDK). What is the OS doing on the cores that DPDK runs on? I mean we here can clean a processor of all activities and are able to run for a long time without any interruptions. Why

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 12 May 2017, Marcelo Tosatti wrote: > In our case, vmstat updates are very rare (CPU is dominated by DPDK). What is the OS doing on the cores that DPDK runs on? I mean we here can clean a processor of all activities and are able to run for a long time without any interruptions. Why

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 12 May 2017, Marcelo Tosatti wrote: > OK, i'll check if the patches from Chris work for us and then add > Tested-by on that. You may not need those if the quiet_vmstat() function is enough for you.

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 12 May 2017, Marcelo Tosatti wrote: > OK, i'll check if the patches from Chris work for us and then add > Tested-by on that. You may not need those if the quiet_vmstat() function is enough for you.

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, May 12, 2017 at 10:11:14AM -0500, Christoph Lameter wrote: > On Fri, 12 May 2017, Marcelo Tosatti wrote: > > > > A bit confused by this one. The vmstat worker is already disabled if there > > > are no updates. Also the patches by Chris Metcalf on data plane mode add a > > > prctl to quiet

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, May 12, 2017 at 10:11:14AM -0500, Christoph Lameter wrote: > On Fri, 12 May 2017, Marcelo Tosatti wrote: > > > > A bit confused by this one. The vmstat worker is already disabled if there > > > are no updates. Also the patches by Chris Metcalf on data plane mode add a > > > prctl to quiet

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 12 May 2017, Marcelo Tosatti wrote: > > A bit confused by this one. The vmstat worker is already disabled if there > > are no updates. Also the patches by Chris Metcalf on data plane mode add a > > prctl to quiet the vmstat workers. > > > > Why do we need more than this? > > If there are

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Fri, 12 May 2017, Marcelo Tosatti wrote: > > A bit confused by this one. The vmstat worker is already disabled if there > > are no updates. Also the patches by Chris Metcalf on data plane mode add a > > prctl to quiet the vmstat workers. > > > > Why do we need more than this? > > If there are

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Thu, May 11, 2017 at 10:37:07AM -0500, Christoph Lameter wrote: > On Tue, 2 May 2017, Luiz Capitulino wrote: > > > Ah, OK. Got this now. I'll give this patch a try. But I think we want > > to hear from Christoph (who worked on reducing the vmstat interruptions > > in the past). > > A bit

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Thu, May 11, 2017 at 10:37:07AM -0500, Christoph Lameter wrote: > On Tue, 2 May 2017, Luiz Capitulino wrote: > > > Ah, OK. Got this now. I'll give this patch a try. But I think we want > > to hear from Christoph (who worked on reducing the vmstat interruptions > > in the past). > > A bit

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, 2 May 2017, Luiz Capitulino wrote: > Ah, OK. Got this now. I'll give this patch a try. But I think we want > to hear from Christoph (who worked on reducing the vmstat interruptions > in the past). A bit confused by this one. The vmstat worker is already disabled if there are no updates.

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, 2 May 2017, Luiz Capitulino wrote: > Ah, OK. Got this now. I'll give this patch a try. But I think we want > to hear from Christoph (who worked on reducing the vmstat interruptions > in the past). A bit confused by this one. The vmstat worker is already disabled if there are no updates.

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, May 02, 2017 at 01:15:27PM -0400, Luiz Capitulino wrote: > On Tue, 2 May 2017 13:52:00 -0300 > Marcelo Tosatti wrote: > > > > I have several questions about the tunables: > > > > > > - What does the vmstat_threshold value mean? What are the implications > > >of

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, May 02, 2017 at 01:15:27PM -0400, Luiz Capitulino wrote: > On Tue, 2 May 2017 13:52:00 -0300 > Marcelo Tosatti wrote: > > > > I have several questions about the tunables: > > > > > > - What does the vmstat_threshold value mean? What are the implications > > >of changing this value?

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, 2 May 2017 13:52:00 -0300 Marcelo Tosatti wrote: > > I have several questions about the tunables: > > > > - What does the vmstat_threshold value mean? What are the implications > >of changing this value? What's the difference in choosing 1, 2, 3 > >or 500?

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, 2 May 2017 13:52:00 -0300 Marcelo Tosatti wrote: > > I have several questions about the tunables: > > > > - What does the vmstat_threshold value mean? What are the implications > >of changing this value? What's the difference in choosing 1, 2, 3 > >or 500? > > Its the

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, May 02, 2017 at 10:28:36AM -0400, Luiz Capitulino wrote: > On Tue, 25 Apr 2017 10:57:19 -0300 > Marcelo Tosatti wrote: > > > The per-CPU vmstat worker is a problem on -RT workloads (because > > ideally the CPU is entirely reserved for the -RT app, without > >

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, May 02, 2017 at 10:28:36AM -0400, Luiz Capitulino wrote: > On Tue, 25 Apr 2017 10:57:19 -0300 > Marcelo Tosatti wrote: > > > The per-CPU vmstat worker is a problem on -RT workloads (because > > ideally the CPU is entirely reserved for the -RT app, without > > interference). The worker

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, 25 Apr 2017 10:57:19 -0300 Marcelo Tosatti wrote: > The per-CPU vmstat worker is a problem on -RT workloads (because > ideally the CPU is entirely reserved for the -RT app, without > interference). The worker transfers accumulated per-CPU > vmstat counters to global

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, 25 Apr 2017 10:57:19 -0300 Marcelo Tosatti wrote: > The per-CPU vmstat worker is a problem on -RT workloads (because > ideally the CPU is entirely reserved for the -RT app, without > interference). The worker transfers accumulated per-CPU > vmstat counters to global counters. This is a

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, Apr 25, 2017 at 03:29:06PM -0400, Rik van Riel wrote: > On Tue, 2017-04-25 at 10:57 -0300, Marcelo Tosatti wrote: > > The per-CPU vmstat worker is a problem on -RT workloads (because > > ideally the CPU is entirely reserved for the -RT app, without > > interference). The worker transfers

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, Apr 25, 2017 at 03:29:06PM -0400, Rik van Riel wrote: > On Tue, 2017-04-25 at 10:57 -0300, Marcelo Tosatti wrote: > > The per-CPU vmstat worker is a problem on -RT workloads (because > > ideally the CPU is entirely reserved for the -RT app, without > > interference). The worker transfers

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, 2017-04-25 at 10:57 -0300, Marcelo Tosatti wrote: > The per-CPU vmstat worker is a problem on -RT workloads (because > ideally the CPU is entirely reserved for the -RT app, without > interference). The worker transfers accumulated per-CPU  > vmstat counters to global counters. > > To

Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

On Tue, 2017-04-25 at 10:57 -0300, Marcelo Tosatti wrote: > The per-CPU vmstat worker is a problem on -RT workloads (because > ideally the CPU is entirely reserved for the -RT app, without > interference). The worker transfers accumulated per-CPU  > vmstat counters to global counters. > > To

[patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

The per-CPU vmstat worker is a problem on -RT workloads (because ideally the CPU is entirely reserved for the -RT app, without interference). The worker transfers accumulated per-CPU vmstat counters to global counters. To resolve the problem, create two tunables: * Userspace configurable

[patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

The per-CPU vmstat worker is a problem on -RT workloads (because ideally the CPU is entirely reserved for the -RT app, without interference). The worker transfers accumulated per-CPU vmstat counters to global counters. To resolve the problem, create two tunables: * Userspace configurable