Re: [PATCH 3/4] filemap: drop the mmap_sem for all blocking operations

2018-12-05 Thread Johannes Weiner
t the fault handler makes things a lot simpler than the -EAGAIN and wait_on_page_locked dance from earlier versions. Nice. Acked-by: Johannes Weiner

Re: [PATCH 2/4] filemap: kill page_cache_read usage in filemap_fault

2018-12-05 Thread Johannes Weiner
cache case significantly. > > Signed-off-by: Josef Bacik That's a great simplification. Looks correct to me. Acked-by: Johannes Weiner

Re: [PATCH] psi: fix reference to kernel commandline enable

2018-12-03 Thread Johannes Weiner
ling/enabling easier for vendor kernels") > Signed-off-by: Baruch Siach Doh, thanks Baruch. Acked-by: Johannes Weiner > --- > init/Kconfig | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/init/Kconfig b/init/Kconfig > index cf5b5a0dcbc2..ed93525

Re: Hackbench pipes regression bisected to PSI

2018-11-27 Thread Johannes Weiner
On Tue, Nov 27, 2018 at 11:46:17AM -0500, Johannes Weiner wrote: > From 347b69a52d1ec7e71df1108cbc5703d6dd0616ba Mon Sep 17 00:00:00 2001 > From: Johannes Weiner > Date: Mon, 26 Nov 2018 09:39:23 -0500 > Subject: [PATCH] psi: make disabling/enabling easier for vendor kernels >

Re: Hackbench pipes regression bisected to PSI

2018-11-27 Thread Johannes Weiner
ed-by: Mel Gorman Thanks for testing it. Let's add these results to the changelog: --- >From 347b69a52d1ec7e71df1108cbc5703d6dd0616ba Mon Sep 17 00:00:00 2001 From: Johannes Weiner Date: Mon, 26 Nov 2018 09:39:23 -0500 Subject: [PATCH] psi: make disabling/enabling easier for vendor kernels Mel Gorman reports a

Re: Hackbench pipes regression bisected to PSI

2018-11-26 Thread Johannes Weiner
On Mon, Nov 26, 2018 at 04:54:47PM +, Mel Gorman wrote: > On Mon, Nov 26, 2018 at 11:07:24AM -0500, Johannes Weiner wrote: > > @@ -509,6 +509,15 @@ config PSI > > > > Say N if unsure. > > > > +config PSI_DEFAULT_DISABLED > > + bool "Req

Re: Hackbench pipes regression bisected to PSI

2018-11-26 Thread Johannes Weiner
ast good commit: eb414681d5a07d28d2ff90dc05f69ec6b232ebd2 > First bad commit: 2ce7135adc9ad081aa3c49744144376ac74fea60 > From 2ce7135adc9ad081aa3c49744144376ac74fea60 Mon Sep 17 00:00:00 2001 > From: Johannes Weiner > Date: Fri, 26 Oct 2018 15:06:31 -0700 > Subject: [PATCH

Re: [PATCH] psi: simplify cgroup_move_task

2018-11-06 Thread Johannes Weiner
; > Warning was: > > kernel/sched/psi.c: In function ‘cgroup_move_task’: > kernel/sched/psi.c:639:13: warning: ‘rq’ may be used uninitialized in this > function [-Wmaybe-uninitialized] > > Fixes: 2ce7135adc9ad ("psi: cgroup support") > Cc: Johannes Weiner > S

Re: [RFC PATCH 2/2] memcg: do not report racy no-eligible OOM tasks

2018-10-26 Thread Johannes Weiner
On Mon, Oct 22, 2018 at 09:13:23AM +0200, Michal Hocko wrote: > From: Michal Hocko > > Tetsuo has reported [1] that a single process group memcg might easily > swamp the log with no-eligible oom victim reports due to race between > the memcg charge and oom_reaper > > Thread 1

Re: [PATCH 0/9] psi: pressure stall information for CPU, memory, and IO v4

2018-10-23 Thread Johannes Weiner
On Thu, Oct 18, 2018 at 07:07:10PM -0700, Andrew Morton wrote: > On Tue, 28 Aug 2018 13:22:49 -0400 Johannes Weiner wrote: > > > This version 4 of the PSI series incorporates feedback from Peter and > > fixes two races in the lockless aggregator that Suren found in his >

Re: [RFC PATCH] memcg, oom: throttle dump_header for memcg ooms without eligible tasks

2018-10-13 Thread Johannes Weiner
On Sat, Oct 13, 2018 at 08:09:30PM +0900, Tetsuo Handa wrote: > -- Michal's patch -- > > 73133 lines (5.79MB) of kernel messages per one run > > [root@ccsecurity ~]# time ./a.out > > real3m44.389s > user0m0.000s > sys 3m42.334s > > [root@ccsecurity ~]# time ./a.out

Re: [RFC PATCH] memcg, oom: throttle dump_header for memcg ooms without eligible tasks

2018-10-12 Thread Johannes Weiner
; information to go on already. > > > > Yeah. Unless we have a storm coming from many different cgroups in > > parallel. But even then we have the allocation context for each OOM so > > we are not losing everything. Should we ever tune this, it can be done > > later with

Re: [RFC PATCH] memcg, oom: throttle dump_header for memcg ooms without eligible tasks

2018-10-12 Thread Johannes Weiner
.ne.jp > Cc: rient...@google.com > Cc: yan...@alibaba-inc.com > Signed-off-by: Michal Hocko So not more than 10 dumps in each 5s interval. That looks reasonable to me. By the time it starts dropping data you have more than enough information to go on already. Acked-by: Johannes Weiner

Re: [PATCH 4/4] mm: zero-seek shrinkers

2018-10-10 Thread Johannes Weiner
On Wed, Oct 10, 2018 at 01:03:50AM +, Rik van Riel wrote: > On Tue, 2018-10-09 at 14:47 -0400, Johannes Weiner wrote: > > > These workloads also deal with tens of thousands of open files and > > use > > /proc for introspection, which ends up growing the proc_inode_c

Re: [PATCH 3/4] mm: workingset: add vmstat counter for shadow nodes

2018-10-10 Thread Johannes Weiner
On Tue, Oct 09, 2018 at 03:08:45PM -0700, Andrew Morton wrote: > On Tue, 9 Oct 2018 14:47:32 -0400 Johannes Weiner wrote: > > > --- a/mm/workingset.c > > +++ b/mm/workingset.c > > @@ -378,11 +378,17 @@ void workingset_update_node(struct xa_node *node) > >

Re: [PATCH 3/4] mm: workingset: add vmstat counter for shadow nodes

2018-10-10 Thread Johannes Weiner
On Tue, Oct 09, 2018 at 03:04:01PM -0700, Andrew Morton wrote: > On Tue, 9 Oct 2018 14:47:32 -0400 Johannes Weiner wrote: > > > Make it easier to catch bugs in the shadow node shrinker by adding a > > counter for the shadow nodes in circulation. > > > >

[PATCH 1/4] mm: workingset: don't drop refault information prematurely fix

2018-10-09 Thread Johannes Weiner
cgroup size. Signed-off-by: Johannes Weiner --- mm/workingset.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/workingset.c b/mm/workingset.c index 1d111913929d..e5c70bc94077 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -418,9 +418,15 @@ static

[PATCH 3/4] mm: workingset: add vmstat counter for shadow nodes

2018-10-09 Thread Johannes Weiner
Make it easier to catch bugs in the shadow node shrinker by adding a counter for the shadow nodes in circulation. Signed-off-by: Johannes Weiner --- include/linux/mmzone.h | 1 + mm/vmstat.c| 1 + mm/workingset.c| 12 ++-- 3 files changed, 12 insertions(+), 2

[PATCH 2/4] mm: workingset: use cheaper __inc_lruvec_state in irqsafe node reclaim

2018-10-09 Thread Johannes Weiner
No need to use the preemption-safe lruvec state function inside the reclaim region that has irqs disabled. Signed-off-by: Johannes Weiner --- mm/workingset.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/workingset.c b/mm/workingset.c index e5c70bc94077..f564aaa6b71d

[PATCH 4/4] mm: zero-seek shrinkers

2018-10-09 Thread Johannes Weiner
zero-seek setting. Reported-by: Domas Mituzas Signed-off-by: Johannes Weiner --- fs/kernfs/mount.c | 3 +++ fs/proc/root.c| 3 +++ mm/vmscan.c | 15 --- mm/workingset.c | 2 +- 4 files changed, 19 insertions(+), 4 deletions(-) diff --git a/fs/kernfs/mount.c b/fs/kern

[PATCH 0/4] mm: workingset & shrinker fixes

2018-10-09 Thread Johannes Weiner
Hi Andrew, these patches address problems we've had in our fleet with excessive shadow radix tree nodes and proc inodes. Patch #1 is a fix for the same-named patch already queued up in the -mm tree. The other three patches are stand-alone.

Re: [PATCH v2] mm: don't raise MEMCG_OOM event due to failed high-order allocation

2018-10-08 Thread Johannes Weiner
eferred to return ENOMEM. > > In this case the idea of raising MEMCG_OOM looks dubious. > > Fix this by moving MEMCG_OOM raising to mem_cgroup_oom() after > allocation order check, so that the event won't be raised for high > order allocations. This change doesn't affect regular page

Re: [PATCH] memcg: remove memcg_kmem_skip_account

2018-09-19 Thread Johannes Weiner
ache_enqueue(). > > Signed-off-by: Shakeel Butt Acked-by: Johannes Weiner

Re: [PATCH 4/9] sched: loadavg: consolidate LOAD_INT, LOAD_FRAC, CALC_LOAD

2018-09-12 Thread Johannes Weiner
On Wed, Sep 12, 2018 at 04:28:28PM -0700, Andrew Morton wrote: > On Tue, 28 Aug 2018 13:22:53 -0400 Johannes Weiner wrote: > > > There are several definitions of those functions/macros in places that > > mess with fixed-point load averages. Provide an official versio

Re: mmotm 2018-09-12-16-40 uploaded (psi)

2018-09-12 Thread Johannes Weiner
/sched.h:1367:0, > from ../kernel/sched/core.c:8: > ../kernel/sched/stats.h: In function 'psi_task_tick': > ../kernel/sched/stats.h:135:33: error: 'struct rq' has no member named 'cpu' >psi_memstall_tick(rq->curr, rq->cpu); This needs to use the SMP/UP config-aware accessor

Re: [PATCH RFC] mm: don't raise MEMCG_OOM event due to failed high-order allocation

2018-09-11 Thread Johannes Weiner
On Mon, Sep 10, 2018 at 02:56:22PM -0700, Roman Gushchin wrote: > The memcg OOM killer is never invoked due to a failed high-order > allocation, however the MEMCG_OOM event can be easily raised. Wasn't the same also true for kernel allocations until recently? We'd signal MEMCG_OOM and then return

Re: + mm-slab-shorten-kmalloc-cache-names-for-large-sizes.patch added to -mm tree

2018-09-08 Thread Johannes Weiner
On Fri, Sep 07, 2018 at 10:58:16PM +0300, Alexey Dobriyan wrote: > On Thu, Sep 06, 2018 at 03:48:49PM -0700, a...@linux-foundation.org wrote: > > - {"kmalloc-1024", 1024}, {"kmalloc-2048", 2048}, > > - {"kmalloc-4096", 4096}, {"kmalloc-8192",

Re: [PATCH] mm: memcontrol: print proper OOM header when no eligible victim left

2018-09-08 Thread Johannes Weiner
On Sat, Sep 08, 2018 at 10:36:06PM +0900, Tetsuo Handa wrote: > On 2018/08/22 1:04, Johannes Weiner wrote: > > When the memcg OOM killer runs out of killable tasks, it currently > > prints a WARN with no further OOM context. This has caused some user > > confusion. >

Re: [PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-09-07 Thread Johannes Weiner
On Fri, Sep 07, 2018 at 04:58:58PM +0200, Peter Zijlstra wrote: > On Fri, Sep 07, 2018 at 10:44:22AM -0400, Johannes Weiner wrote: > > > > This does the whole seqcount thing 6x, which is a bit of a waste. > > > > [...] > > > > > It's a bit cumbe

Re: [PATCH 0/9] psi: pressure stall information for CPU, memory, and IO v4

2018-09-07 Thread Johannes Weiner
On Fri, Sep 07, 2018 at 01:04:07PM +0200, Peter Zijlstra wrote: > So yeah, grudingly acked. Did you want me to pick this up through the > scheduler tree since most of this lives there? Thanks for the ack. As for routing it, I'll leave that decision to you and Andrew. It touches stuff all over,

Re: [PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-09-07 Thread Johannes Weiner
On Fri, Sep 07, 2018 at 12:24:58PM +0200, Peter Zijlstra wrote: > On Tue, Aug 28, 2018 at 01:22:57PM -0400, Johannes Weiner wrote: > > +static void psi_clock(struct work_struct *work) > > +{ > > + struct delayed_work *dwork; > > + struct psi_group *

Re: [PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-09-07 Thread Johannes Weiner
On Fri, Sep 07, 2018 at 12:16:34PM +0200, Peter Zijlstra wrote: > On Tue, Aug 28, 2018 at 01:22:57PM -0400, Johannes Weiner wrote: > > +enum psi_states { > > + PSI_IO_SOME, > > + PSI_IO_FULL, > > + PSI_MEM_SOME, > > + PSI_MEM_FULL, > > + PSI_CPU_

Re: [PATCH 0/9] psi: pressure stall information for CPU, memory, and IO v4

2018-09-05 Thread Johannes Weiner
On Tue, Aug 28, 2018 at 01:22:49PM -0400, Johannes Weiner wrote: > This version 4 of the PSI series incorporates feedback from Peter and > fixes two races in the lockless aggregator that Suren found in his > testing and which caused the sample calculation to sometimes underflow >

Re: [PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-08-28 Thread Johannes Weiner
On Tue, Aug 28, 2018 at 01:11:11PM -0700, Randy Dunlap wrote: > On 08/28/2018 10:22 AM, Johannes Weiner wrote: > > diff --git a/Documentation/accounting/psi.txt > > b/Documentation/accounting/psi.txt > > new file mode 100644 > > index ..51e7ef14142e

[PATCH 3/9] delayacct: track delays from thrashing cache pages

2018-08-28 Thread Johannes Weiner
0ms Signed-off-by: Johannes Weiner --- include/linux/delayacct.h | 23 +++ include/uapi/linux/taskstats.h | 6 +- kernel/delayacct.c | 15 +++ mm/filemap.c | 11 +++ tools/accounting/getdelays.c | 8

[PATCH 0/9] psi: pressure stall information for CPU, memory, and IO v4

2018-08-28 Thread Johannes Weiner
This version 4 of the PSI series incorporates feedback from Peter and fixes two races in the lockless aggregator that Suren found in his testing and which caused the sample calculation to sometimes underflow and record bogusly large samples; details at the bottom of this email.

[PATCH 4/9] sched: loadavg: consolidate LOAD_INT, LOAD_FRAC, CALC_LOAD

2018-08-28 Thread Johannes Weiner
There are several definitions of those functions/macros in places that mess with fixed-point load averages. Provide an official version. Signed-off-by: Johannes Weiner --- .../platforms/cell/cpufreq_spudemand.c| 2 +- arch/powerpc/platforms/cell/spufs/sched.c | 9 +++- arch

[PATCH 6/9] sched: sched.h: make rq locking and clock functions available in stats.h

2018-08-28 Thread Johannes Weiner
kernel/sched/sched.h includes "stats.h" half-way through the file. The next patch introduces users of sched.h's rq locking functions and update_rq_clock() in kernel/sched/stats.h. Move those definitions up in the file so they are available in stats.h. Signed-off-by: Johannes Weiner -

[PATCH 1/9] mm: workingset: don't drop refault information prematurely

2018-08-28 Thread Johannes Weiner
From: Johannes Weiner If we keep just enough refault information to match the *current* page cache during reclaim time, we could lose a lot of events when there is only a temporary spike in non-cache memory consumption that pushes out all the cache. Once cache comes back, we won't see those

[PATCH 5/9] sched: loadavg: make calc_load_n() public

2018-08-28 Thread Johannes Weiner
It's going to be used in a later patch. Keep the churn separate. Signed-off-by: Johannes Weiner --- include/linux/sched/loadavg.h | 3 + kernel/sched/loadavg.c| 138 +- 2 files changed, 72 insertions(+), 69 deletions(-) diff --git a/include/linux/sched

[PATCH 9/9] psi: cgroup support

2018-08-28 Thread Johannes Weiner
checks outward - factor out iterate_groups() Acked-by: Tejun Heo Signed-off-by: Johannes Weiner --- Documentation/accounting/psi.txt| 9 ++ Documentation/admin-guide/cgroup-v2.rst | 18 include/linux/cgroup-defs.h | 4 + include/linux/cgroup.h | 15

[PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-08-28 Thread Johannes Weiner
underflow into bogusly large time deltas, as per Suren - fix rare accounting artifacts from CPU hotplugging, as per Peter - make the aggregation loop over all states more readable, as per Peter Signed-off-by: Johannes Weiner --- Documentation/accounting/psi.txt | 64 +++ include/linux/psi.h

[PATCH 7/9] sched: introduce this_rq_lock_irq()

2018-08-28 Thread Johannes Weiner
do_sched_yield() disables IRQs, looks up this_rq() and locks it. The next patch is adding another site with the same pattern, so provide a convenience function for it. Signed-off-by: Johannes Weiner --- kernel/sched/core.c | 4 +--- kernel/sched/sched.h | 12 2 files changed, 13

[PATCH 2/9] mm: workingset: tell cache transitions from workingset thrashing

2018-08-28 Thread Johannes Weiner
r 7 sparsemem section bits. v4: - fix a typo in the comments, as per Suren Signed-off-by: Johannes Weiner --- include/linux/mmzone.h | 1 + include/linux/page-flags.h | 5 +- include/linux/swap.h | 2 +- include/trace/events/mmflags.h | 1 + mm/filemap.c |

Re: [PATCH v2 1/3] mm: rework memcg kernel stack accounting

2018-08-24 Thread Johannes Weiner
On Thu, Aug 23, 2018 at 09:23:50AM -0700, Roman Gushchin wrote: > On Wed, Aug 22, 2018 at 04:12:13PM +0200, Michal Hocko wrote: > > On Tue 21-08-18 14:35:57, Roman Gushchin wrote: > > > @@ -248,9 +253,20 @@ static unsigned long *alloc_thread_stack_node(struct > > > task_struct *tsk, int node) > >

Re: [PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-08-22 Thread Johannes Weiner
On Wed, Aug 22, 2018 at 11:10:24AM +0200, Peter Zijlstra wrote: > On Tue, Aug 21, 2018 at 04:11:15PM -0400, Johannes Weiner wrote: > > On Fri, Aug 03, 2018 at 07:21:39PM +0200, Peter Zijlstra wrote: > > > On Wed, Aug 01, 2018 at 11:19:57AM -0400, Joh

Re: [PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-08-21 Thread Johannes Weiner
On Fri, Aug 03, 2018 at 07:21:39PM +0200, Peter Zijlstra wrote: > On Wed, Aug 01, 2018 at 11:19:57AM -0400, Johannes Weiner wrote: > > + time = READ_ONCE(groupc->times[s]); > > + /* > > +* In addition to alre

Re: [PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-08-21 Thread Johannes Weiner
Hi, a quick update on that feedback before I send out v4: On Fri, Aug 03, 2018 at 06:56:41PM +0200, Peter Zijlstra wrote: > On Wed, Aug 01, 2018 at 11:19:57AM -0400, Johannes Weiner wrote: > > +static bool test_state(unsigned int *tasks, int cpu, enum psi_states state) > > +{

Re: [PATCH 2/2] memcg, oom: emit oom report when there is no eligible task

2018-08-21 Thread Johannes Weiner
Wed 08-08-18 10:45:15, Johannes Weiner wrote: > [...] > > > >From bba01122f739b05a689dbf1eeeb4f0e07affd4e7 Mon Sep 17 00:00:00 2001 > > > From: Johannes Weiner > > > Date: Wed, 8 Aug 2018 09:59:40 -0400 > > > Subject: [PATCH] mm: memcontrol:

Re: [PATCH RFC] mm: don't miss the last page because of round-off error

2018-08-21 Thread Johannes Weiner
On Tue, Aug 21, 2018 at 08:11:44AM +0300, Konstantin Khlebnikov wrote: > On Sat, Aug 18, 2018 at 4:22 AM, Matthew Wilcox wrote: > > On Fri, Aug 17, 2018 at 04:18:34PM -0700, Roman Gushchin wrote: > >> - scan = div64_u64(scan * fraction[file], > >> -

[PATCH] mm: memcontrol: print proper OOM header when no eligible victim left

2018-08-21 Thread Johannes Weiner
-off-by: Johannes Weiner Acked-by: Michal Hocko --- mm/memcontrol.c | 2 -- mm/oom_kill.c | 13 ++--- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4e3c1315b1de..29d9d1a69b36 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c

Re: [PATCH v8 0/2] Directed kmem charging

2018-08-17 Thread Johannes Weiner
feedback I remember giving. [ My only gripe really is that it applies current->active_memcg only to kmem charges, not others as well. Right now it doesn't matter, but I can see this costing a kernel developer implementing remote charges for something other than kmem some time to realize. ] Anyway, please feel free to add Acked-by: Johannes Weiner for 1/2 and 2/2 plus their two fixlets.

Re: [RFC PATCH 1/2] mm: rework memcg kernel stack accounting

2018-08-15 Thread Johannes Weiner
On Wed, Aug 15, 2018 at 09:55:17AM -0700, Roman Gushchin wrote: > On Wed, Aug 15, 2018 at 12:39:23PM -0400, Johannes Weiner wrote: > > On Tue, Aug 14, 2018 at 05:36:19PM -0700, Roman Gushchin wrote: > > > @@ -224,9 +224,14 @@ static unsigned long *alloc_thread_stack_node(struct

Re: [RFC PATCH 1/2] mm: rework memcg kernel stack accounting

2018-08-15 Thread Johannes Weiner
On Tue, Aug 14, 2018 at 05:36:19PM -0700, Roman Gushchin wrote: > @@ -224,9 +224,14 @@ static unsigned long *alloc_thread_stack_node(struct > task_struct *tsk, int node) > return s->addr; > } > > + /* > + * Allocated stacks are cached and later reused by new

Re: [PATCH RFC 1/3] cgroup: list all subsystem states in debugfs files

2018-08-13 Thread Johannes Weiner
On Mon, Aug 13, 2018 at 06:48:42AM -0700, Tejun Heo wrote: > Hello, Konstantin. > > On Mon, Aug 13, 2018 at 09:58:05AM +0300, Konstantin Khlebnikov wrote: > > After removing cgroup subsystem state could leak or live in background > > forever because it is pinned by some reference. For example

Re: WARNING in try_charge

2018-08-09 Thread Johannes Weiner
On Thu, Aug 09, 2018 at 10:57:43PM +0900, Tetsuo Handa wrote: > From b1f38168f14397c7af9c122cd8207663d96e02ec Mon Sep 17 00:00:00 2001 > From: Tetsuo Handa > Date: Thu, 9 Aug 2018 22:49:40 +0900 > Subject: [PATCH] mm, oom: task_will_free_mem(current) should retry until > memory reserve fails >

Re: [PATCH 2/2] memcg, oom: emit oom report when there is no eligible task

2018-08-08 Thread Johannes Weiner
back trace of the offending path but also the full > memcg state - memory counters and existing tasks. > > Suggested-by: Johannes Weiner > Signed-off-by: Michal Hocko > --- > include/linux/oom.h | 2 ++ > mm/memcontrol.c | 24 +--- > mm/oom_kil

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-07 Thread Johannes Weiner
On Tue, Aug 07, 2018 at 10:23:32PM +0200, Michal Hocko wrote: > On Tue 07-08-18 16:02:47, Johannes Weiner wrote: > > On Tue, Aug 07, 2018 at 09:25:53AM +0200, Michal Hocko wrote: > > > From: Michal Hocko > > > > > > "memcg, oom: move out_of_

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-07 Thread Johannes Weiner
On Tue, Aug 07, 2018 at 07:15:11PM +0900, Tetsuo Handa wrote: > On 2018/08/07 16:25, Michal Hocko wrote: > > @@ -1703,7 +1703,8 @@ static enum oom_status mem_cgroup_oom(struct > > mem_cgroup *memcg, gfp_t mask, int > > return OOM_ASYNC; > > } > > > > - if

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-07 Thread Johannes Weiner
On Tue, Aug 07, 2018 at 09:25:53AM +0200, Michal Hocko wrote: > From: Michal Hocko > > "memcg, oom: move out_of_memory back to the charge path" has added a > warning triggered when the oom killer cannot find any eligible task > and so there is no way to reclaim the oom memcg under its hard

Re: [PATCH 0/9] psi: pressure stall information for CPU, memory, and IO v3

2018-08-07 Thread Johannes Weiner
On Tue, Aug 07, 2018 at 01:50:09PM +0200, peter enderborg wrote: > On 08/01/2018 05:19 PM, Johannes Weiner wrote: > > > > A kernel with CONFIG_PSI=y will create a /proc/pressure directory with > > 3 files: cpu, memory, and io. If using cgroup2, cgroups will also

Re: [PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-08-06 Thread Johannes Weiner
On Mon, Aug 06, 2018 at 05:25:28PM +0200, Peter Zijlstra wrote: > On Mon, Aug 06, 2018 at 11:05:50AM -0400, Johannes Weiner wrote: > > Argh, that's right. This needs an explicit count if we want to access > > it locklessly. And you already said you didn't like that this is the

Re: WARNING in try_charge

2018-08-06 Thread Johannes Weiner
On Mon, Aug 06, 2018 at 04:21:24PM +0200, Michal Hocko wrote: > On Mon 06-08-18 13:57:38, Dmitry Vyukov wrote: > > On Mon, Aug 6, 2018 at 1:02 PM, Michal Hocko wrote: > > > If you have a strong reason to believe that this is an abuse of WARN I > > > am all happy to change that. But I haven't

Re: [PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-08-06 Thread Johannes Weiner
On Fri, Aug 03, 2018 at 07:07:33PM +0200, Peter Zijlstra wrote: > On Wed, Aug 01, 2018 at 11:19:57AM -0400, Johannes Weiner wrote: > > +static bool psi_update_stats(struct psi_group *group) > > +{ > > + u64 deltas[NR_PSI_STATES - 1] = { 0, }; > > + unsig

Re: [PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-08-06 Thread Johannes Weiner
On Fri, Aug 03, 2018 at 06:56:41PM +0200, Peter Zijlstra wrote: > On Wed, Aug 01, 2018 at 11:19:57AM -0400, Johannes Weiner wrote: > > +static bool psi_update_stats(struct psi_group *group) > > +{ > > + u64 deltas[NR_PSI_STATES - 1] = { 0, }; > > + unsig

Re: [PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-08-06 Thread Johannes Weiner
On Fri, Aug 03, 2018 at 06:56:41PM +0200, Peter Zijlstra wrote: > On Wed, Aug 01, 2018 at 11:19:57AM -0400, Johannes Weiner wrote: > > +static bool test_state(unsigned int *tasks, int cpu, enum psi_states state) > > +{ > > + switch (state) { > > + case PSI_IO_SOME: &

Re: [PATCH 2/9] mm: workingset: tell cache transitions from workingset thrashing

2018-08-02 Thread Johannes Weiner
On Thu, Aug 02, 2018 at 08:57:31AM +0200, peter enderborg wrote: > On 08/01/2018 05:13 PM, Johannes Weiner wrote: > > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h > > index e34a27727b9a..7af1c3c15d8e 100644 > > --- a/include/linux/page-flags.h > &g

Re: [PATCH 2/9] mm: workingset: tell cache transitions from workingset thrashing

2018-08-02 Thread Johannes Weiner
Hi Suren, On Wed, Aug 01, 2018 at 02:56:27PM -0700, Suren Baghdasaryan wrote: > On Wed, Aug 1, 2018 at 8:19 AM, Johannes Weiner wrote: > > /* > > -* The unsigned subtraction here gives an accurate distance > > -* across inactive_age ov

Re: [PATCH 3/3] mm, oom: introduce memory.oom.group

2018-08-01 Thread Johannes Weiner
n and are never killed. > > This patch doesn't change the OOM victim selection algorithm. > > Signed-off-by: Roman Gushchin > Cc: Michal Hocko > Cc: Johannes Weiner > Cc: David Rientjes > Cc: Tetsuo Handa > Cc: Tejun Heo The semantics make sense to me and the code is straight-forward. With Michal's other feedback incorporated, please feel free to add: Acked-by: Johannes Weiner

Re: [PATCH 3/3] mm, oom: introduce memory.oom.group

2018-08-01 Thread Johannes Weiner
On Wed, Aug 01, 2018 at 07:55:03AM +0200, Michal Hocko wrote: > On Tue 31-07-18 18:14:48, Roman Gushchin wrote: > > On Tue, Jul 31, 2018 at 11:07:00AM +0200, Michal Hocko wrote: > > > On Mon 30-07-18 11:01:00, Roman Gushchin wrote: > > > > +struct mem_cgroup *mem_cgroup_get_oom_group(struct

Re: [PATCH 2/3] mm, oom: refactor oom_kill_process()

2018-08-01 Thread Johannes Weiner
gt; Acked-by: Michal Hocko > Acked-by: Johannes Weiner > Acked-by: David Rientjes > Cc: Vladimir Davydov > Cc: Tetsuo Handa > Cc: David Rientjes > Cc: Tejun Heo > Cc: Michal Hocko > Signed-off-by: Andrew Morton This is pretty straight-forward. Acked-by: Johannes Weiner

Re: [PATCH 1/3] mm: introduce mem_cgroup_put() helper

2018-08-01 Thread Johannes Weiner
> Signed-off-by: Roman Gushchin > Reviewed-by: Shakeel Butt > Reviewed-by: Andrew Morton > Cc: Shakeel Butt > Cc: Johannes Weiner > Cc: Michal Hocko > Signed-off-by: Andrew Morton > Signed-off-by: Stephen Rothwell Acked-by: Johannes Weiner

Re: [PATCH] memcg: Remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure

2018-08-01 Thread Johannes Weiner
On Tue, Jul 31, 2018 at 04:39:08PM -0700, Andrew Morton wrote: > On Mon, 30 Jul 2018 11:31:13 -0400 Johannes Weiner wrote: > > > Subject: [PATCH] mm: memcontrol: simplify memcg idr allocation and error > > unwinding > > > > The memcg ID is allocated early in

[PATCH 5/9] sched: loadavg: make calc_load_n() public

2018-08-01 Thread Johannes Weiner
It's going to be used in a later patch. Keep the churn separate. Signed-off-by: Johannes Weiner --- include/linux/sched/loadavg.h | 3 + kernel/sched/loadavg.c| 138 +- 2 files changed, 72 insertions(+), 69 deletions(-) diff --git a/include/linux/sched

[PATCH 9/9] psi: cgroup support

2018-08-01 Thread Johannes Weiner
-by: Johannes Weiner --- Documentation/accounting/psi.txt | 9 Documentation/cgroup-v2.txt | 18 +++ include/linux/cgroup-defs.h | 4 ++ include/linux/cgroup.h | 15 ++ include/linux/psi.h | 25 ++ init/Kconfig | 4

[PATCH 4/9] sched: loadavg: consolidate LOAD_INT, LOAD_FRAC, CALC_LOAD

2018-08-01 Thread Johannes Weiner
There are several definitions of those functions/macros in places that mess with fixed-point load averages. Provide an official version. Signed-off-by: Johannes Weiner --- .../platforms/cell/cpufreq_spudemand.c| 2 +- arch/powerpc/platforms/cell/spufs/sched.c | 9 +++- arch

[PATCH 7/9] sched: introduce this_rq_lock_irq()

2018-08-01 Thread Johannes Weiner
do_sched_yield() disables IRQs, looks up this_rq() and locks it. The next patch is adding another site with the same pattern, so provide a convenience function for it. Signed-off-by: Johannes Weiner --- kernel/sched/core.c | 4 +--- kernel/sched/sched.h | 12 2 files changed, 13

[PATCH 6/9] sched: sched.h: make rq locking and clock functions available in stats.h

2018-08-01 Thread Johannes Weiner
kernel/sched/sched.h includes "stats.h" half-way through the file. The next patch introduces users of sched.h's rq locking functions and update_rq_clock() in kernel/sched/stats.h. Move those definitions up in the file so they are available in stats.h. Signed-off-by: Johannes Weiner -

[PATCH 8/9] psi: pressure stall information for CPU, memory, and IO

2018-08-01 Thread Johannes Weiner
un Signed-off-by: Johannes Weiner --- Documentation/accounting/psi.txt | 64 +++ include/linux/psi.h | 27 ++ include/linux/psi_types.h| 87 + include/linux/sched.h| 10 + init/Kconfig | 15 + kernel/fork.c| 4 + ke

[PATCH 2/9] mm: workingset: tell cache transitions from workingset thrashing

2018-08-01 Thread Johannes Weiner
r 7 sparsemem section bits. Signed-off-by: Johannes Weiner --- include/linux/mmzone.h | 1 + include/linux/page-flags.h | 5 +- include/linux/swap.h | 2 +- include/trace/events/mmflags.h | 1 + mm/filemap.c | 9 ++-- mm/huge_memory.c | 1 +

[PATCH 3/9] delayacct: track delays from thrashing cache pages

2018-08-01 Thread Johannes Weiner
0ms Signed-off-by: Johannes Weiner --- include/linux/delayacct.h | 23 +++ include/uapi/linux/taskstats.h | 6 +- kernel/delayacct.c | 15 +++ mm/filemap.c | 11 +++ tools/accounting/getdelays.c | 8

[PATCH 1/9] mm: workingset: don't drop refault information prematurely

2018-08-01 Thread Johannes Weiner
From: Johannes Weiner If we keep just enough refault information to match the CURRENT page cache during reclaim time, we could lose a lot of events when there is only a temporary spike in non-cache memory consumption that pushes out all the cache. Once cache comes back, we won't see those

Re: [PATCH v2] mm: terminate the reclaim early when direct reclaiming

2018-07-31 Thread Johannes Weiner
On Tue, Jul 31, 2018 at 07:09:28PM +0800, Zhaoyang Huang wrote: > This patch try to let the direct reclaim finish earlier than it used > to be. The problem comes from We observing that the direct reclaim > took a long time to finish when memcg is enabled. By debugging, we > find that the reason is

Re: [PATCH 0/3] introduce memory.oom.group

2018-07-31 Thread Johannes Weiner
On Mon, Jul 30, 2018 at 06:49:31PM -0700, David Rientjes wrote: > On Mon, 30 Jul 2018, Roman Gushchin wrote: > > > This is a tiny implementation of cgroup-aware OOM killer, > > which adds an ability to kill a cgroup as a single unit > > and so guarantee the integrity of the workload. > > > >

Re: [PATCH 0/10] psi: pressure stall information for CPU, memory, and IO v2

2018-07-30 Thread Johannes Weiner
On Sat, Jul 28, 2018 at 12:01:23AM +0200, Pavel Machek wrote: > > How do you use this feature? > > > > A kernel with CONFIG_PSI=y will create a /proc/pressure directory with > > 3 files: cpu, memory, and io. If using cgroup2, cgroups will also > > Could we get the config named

Re: [PATCH] memcg: Remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure

2018-07-30 Thread Johannes Weiner
On Sun, Jul 29, 2018 at 10:26:21PM +0300, Vladimir Davydov wrote: > On Fri, Jul 27, 2018 at 03:31:34PM -0400, Johannes Weiner wrote: > > That said, the lifetime of the root reference on the ID is the online > > state, we put that in css_offline. Is there a reason we need to have &

Re: Making direct reclaim fail when thrashing

2018-07-27 Thread Johannes Weiner
On Fri, Jul 27, 2018 at 11:21:43AM -0500, Daniel Drake wrote: > Split from the thread > [PATCH 0/10] psi: pressure stall information for CPU, memory, and IO v2 > where we were discussing if/how to make the direct reclaim codepath > fail if we're excessively thrashing, so that the OOM killer

Re: [PATCH] mm: terminate the reclaim early when direct reclaiming

2018-07-27 Thread Johannes Weiner
Hi Zhaoyang, On Fri, Jul 27, 2018 at 05:19:25PM +0800, Zhaoyang Huang wrote: > This patch try to let the direct reclaim finish earlier than it used > to be. The problem comes from We observing that the direct reclaim > took a long time to finish when memcg is enabled. By debugging, we > find that

Re: [PATCH] memcg: Remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure

2018-07-27 Thread Johannes Weiner
/r/152354470916.22460.14397070748001974638.stgit@localhost.localdomain > Fixes 73f576c04b94 ("mm: memcontrol: fix cgroup creation failure after many > small jobs") > Signed-off-by: Kirill Tkhai > Cc: Johannes Weiner > Cc: Vladimir Davydov > Cc: Michal Hocko > Signed-off

Re: [PATCH 0/10] psi: pressure stall information for CPU, memory, and IO v2

2018-07-26 Thread Johannes Weiner
On Thu, Jul 26, 2018 at 11:07:32AM +1000, Singh, Balbir wrote: > On 7/25/18 1:15 AM, Johannes Weiner wrote: > > On Tue, Jul 24, 2018 at 07:14:02AM +1000, Balbir Singh wrote: > >> Does the mechanism scale? I am a little concerned about how frequently > >> this infrastruct

Re: [PATCH 08/10] psi: pressure stall information for CPU, memory, and IO

2018-07-24 Thread Johannes Weiner
On Thu, Jul 19, 2018 at 10:31:15PM +0200, Peter Zijlstra wrote: > On Thu, Jul 19, 2018 at 02:47:40PM -0400, Johannes Weiner wrote: > > On Wed, Jul 18, 2018 at 02:03:18PM +0200, Peter Zijlstra wrote: > > > On Thu, Jul 12, 2018 at 01:29:40PM -0400, Johannes Weiner wrote: > &g

Re: [PATCH 09/10] psi: cgroup support

2018-07-24 Thread Johannes Weiner
Hi Peter, On Tue, Jul 17, 2018 at 05:40:59PM +0200, Peter Zijlstra wrote: > On Thu, Jul 12, 2018 at 01:29:41PM -0400, Johannes Weiner wrote: > > +/** > > + * cgroup_move_task - move task to a different cgroup > > + * @task: the task > > + * @to: the target css_se

Re: [PATCH 0/10] psi: pressure stall information for CPU, memory, and IO v2

2018-07-24 Thread Johannes Weiner
Hi Balbir, On Tue, Jul 24, 2018 at 07:14:02AM +1000, Balbir Singh wrote: > Does the mechanism scale? I am a little concerned about how frequently > this infrastructure is monitored/read/acted upon. I expect most users to poll in the frequency ballpark of the running averages (10s, 1m, 5m). Our

Re: [PATCH 02/10] mm: workingset: tell cache transitions from workingset thrashing

2018-07-23 Thread Johannes Weiner
On Mon, Jul 23, 2018 at 05:35:35PM +0200, Arnd Bergmann wrote: > On Mon, Jul 23, 2018 at 5:23 PM, Johannes Weiner wrote: > > From 1d24635a6c7cd395bad5c29a3b9e5d2e98d9ab84 Mon Sep 17 00:00:00 2001 > > From: Johannes Weiner > > Date: Mon, 23 Jul 2018 10:18:23 -0400 > >

Re: [PATCH 02/10] mm: workingset: tell cache transitions from workingset thrashing

2018-07-23 Thread Johannes Weiner
Hi Arnd, On Mon, Jul 23, 2018 at 03:36:09PM +0200, Arnd Bergmann wrote: > On Thu, Jul 12, 2018 at 7:29 PM, Johannes Weiner wrote: > > How many page->flags does this leave us with on 32-bit? > > > > 20 bits are always page flags > > > > 21 i

Re: [PATCH 08/10] psi: pressure stall information for CPU, memory, and IO

2018-07-20 Thread Johannes Weiner
On Wed, Jul 18, 2018 at 06:06:23PM -0400, Johannes Weiner wrote: > On Tue, Jul 17, 2018 at 05:01:42PM +0200, Peter Zijlstra wrote: > > On Thu, Jul 12, 2018 at 01:29:40PM -0400, Johannes Weiner wrote: > > > +static bool psi_update_stats(struct psi_group *group) > &g

Re: [PATCH 08/10] psi: pressure stall information for CPU, memory, and IO

2018-07-19 Thread Johannes Weiner
On Wed, Jul 18, 2018 at 02:03:18PM +0200, Peter Zijlstra wrote: > On Thu, Jul 12, 2018 at 01:29:40PM -0400, Johannes Weiner wrote: > > + /* Update task counts according to the set/clear bitmasks */ > > + for (to = 0; (bo = ffs(clear)); to += bo, clear >>= bo) { &g

Re: [PATCH 08/10] psi: pressure stall information for CPU, memory, and IO

2018-07-19 Thread Johannes Weiner
On Thu, Jul 19, 2018 at 08:08:20AM -0700, Linus Torvalds wrote: > On Wed, Jul 18, 2018 at 5:03 AM Peter Zijlstra wrote: > > > > And as said before, we can compress the state from 12 bytes, to 6 bits > > (or 1 byte), giving another 11 bytes for 59 bytes free. > > > > Leaving us just 5 bytes short

Re: [PATCH 08/10] psi: pressure stall information for CPU, memory, and IO

2018-07-19 Thread Johannes Weiner
On Thu, Jul 19, 2018 at 11:26:14AM +0200, Peter Zijlstra wrote: > On Wed, Jul 18, 2018 at 02:03:18PM +0200, Peter Zijlstra wrote: > > > Leaving us just 5 bytes short of needing a single cacheline :/ > > > > struct ponies { > > unsigned int tasks[3];

Re: [PATCH 08/10] psi: pressure stall information for CPU, memory, and IO

2018-07-18 Thread Johannes Weiner
On Wed, Jul 18, 2018 at 02:03:18PM +0200, Peter Zijlstra wrote: > On Thu, Jul 12, 2018 at 01:29:40PM -0400, Johannes Weiner wrote: > > + /* Time in which tasks wait for the CPU */ > > + state = PSI_NONE; > > + if (tasks[NR_RUNNING] > 1) > > + state

  1   2   3   4   5   6   7   8   9   10   >