Re: [PATCH 1/5] mm/workingset: remove unused @mapping argument in workingset_eviction()

2019-02-22 Thread Johannes Weiner
On Fri, Feb 22, 2019 at 08:43:33PM +0300, Andrey Ryabinin wrote: > workingset_eviction() doesn't use and never did use the @mapping argument. > Remove it. > > Signed-off-by: Andrey Ryabinin > Cc: Johannes Weiner > Cc: Michal Hocko > Cc: Vlastimil Babka > Cc: Rik va

Re: [PATCH] mm, oom: OOM killer use rss size without shmem

2019-02-22 Thread Johannes Weiner
On Fri, Feb 22, 2019 at 08:10:01AM +0100, Michal Hocko wrote: > On Fri 22-02-19 13:37:33, Junil Lee wrote: > > The oom killer use get_mm_rss() function to estimate how free memory > > will be reclaimed when the oom killer select victim task. > > > > However, the returned rss size by get_mm_rss()

[PATCH 3/6] mm: memcontrol: replace node summing with memcg_page_state()

2019-02-15 Thread Johannes Weiner
Instead of adding up the node counters, use memcg_page_state() to get the memcg state directly. This is a bit cheaper and more stream-lined. Signed-off-by: Johannes Weiner --- mm/memcontrol.c | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/memcontrol.c b/mm

[PATCH 5/6] mm: memcontrol: push down mem_cgroup_nr_lru_pages()

2019-02-15 Thread Johannes Weiner
mem_cgroup_nr_lru_pages() is just a convenience wrapper around memcg_page_state() that takes bitmasks of lru indexes and aggregates the counts for those. Replace callsites where the bitmask is simple enough with direct memcg_page_state() call(s). Signed-off-by: Johannes Weiner --- mm

[PATCH 6/6] mm: memcontrol: quarantine the mem_cgroup_[node_]nr_lru_pages() API

2019-02-15 Thread Johannes Weiner
Only memcg_numa_stat_show() uses those wrappers and the lru bitmasks, group them together. Signed-off-by: Johannes Weiner --- include/linux/mmzone.h | 5 mm/memcontrol.c| 67 +++--- 2 files changed, 36 insertions(+), 36 deletions(-) diff --git

[PATCH 2/6] mm: memcontrol: replace zone summing with lruvec_page_state()

2019-02-15 Thread Johannes Weiner
Instead of adding up the zone counters, use lruvec_page_state() to get the node state directly. This is a bit cheaper and more stream-lined. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 18 -- mm/memcontrol.c| 2 +- mm/vmscan.c| 2

[PATCH 4/6] mm: memcontrol: push down mem_cgroup_node_nr_lru_pages()

2019-02-15 Thread Johannes Weiner
of mem_cgroup_node_nr_lru_pages(), so make that function private again, too. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 10 -- mm/memcontrol.c| 10 +++--- mm/workingset.c| 5 +++-- 3 files changed, 10 insertions(+), 15 deletions(-) diff --git

[PATCH 0/6] mm: memcontrol: clean up the LRU counts tracking

2019-02-15 Thread Johannes Weiner
The memcg LRU stats usage is currently a bit messy. Memcg has private per-zone counters because reclaim needs zone granularity sometimes, but we also have plenty of users that need to awkwardly sum them up to node or memcg granularity. Meanwhile the canonical per-memcg vmstats do not track the LRU

[PATCH 1/6] mm: memcontrol: track LRU counts in the vmstats array

2019-02-15 Thread Johannes Weiner
-by: Johannes Weiner --- include/linux/mm_inline.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index 04ec454d44ce..6f2fef7b0784 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -29,7 +29,7 @@ static

Re: [PATCH] psi: avoid divide-by-zero crash inside virtual machines

2019-02-14 Thread Johannes Weiner
On Thu, Feb 14, 2019 at 11:58:55AM -0800, Andrew Morton wrote: > On Thu, 14 Feb 2019 14:31:57 -0500 Johannes Weiner wrote: > > > --- a/kernel/sched/psi.c > > +++ b/kernel/sched/psi.c > > @@ -322,7 +322,7 @@ static bool update_stats(struct psi_group *group) > >

[PATCH] psi: avoid divide-by-zero crash inside virtual machines

2019-02-14 Thread Johannes Weiner
ate the elapsed period, the result, our pressure divisor, will be 0. Ouch. Fix this by correctly handling the situation when the elapsed time between aggregation runs is precisely two periods, and advance the expiration timestamp correctly to period into the future. Reported-by: Łukasz Siudut Signed-o

Re: [PATCH v2 0/3] vmalloc enhancements

2019-02-12 Thread Johannes Weiner
On Tue, Feb 12, 2019 at 09:56:45AM -0800, Roman Gushchin wrote: > The patchset contains few changes to the vmalloc code, which are > leading to some performance gains and code simplification. > > Also, it exports a number of pages, used by vmalloc(), > in /proc/meminfo. > > Patch (1) removes

Re: [PATCH] proc, oom: do not report alien mms when setting oom_score_adj

2019-02-12 Thread Johannes Weiner
gt; remove all the tasks crawling as suggested by [2] > > [1] > http://lkml.kernel.org/r/97fce864-6f75-bca5-14bc-12c9f890e...@i-love.sakura.ne.jp > [2] http://lkml.kernel.org/r/20190117155159.ga4...@dhcp22.suse.cz > Reported-by: Tetsuo Handa > Signed-off-by: Michal Hocko Acked-by: Johannes Weiner

Re: [PATCH 0/3] vmalloc enhancements

2019-02-11 Thread Johannes Weiner
These slipped through the cracks. +CC Andrew directly. Andrew, if it's not too late yet, could you consider them for 5.1? On Wed, Dec 19, 2018 at 09:37:48AM -0800, Roman Gushchin wrote: > The patchset contains few changes to the vmalloc code, which are > leading to some performance gains and code

Re: [PATCH v2 2/2] mm: Consider subtrees in memory.events

2019-02-11 Thread Johannes Weiner
nge in behaviour, this can be reverted to the old > behaviour by mounting with the `memory_localevents` flag set. However, > we use the new behaviour by default as there's a lack of evidence that > there are any current users of memory.events that would find this change > undesirable.

Re: [PATCH v2 1/2] mm: Rename ambiguously named memory.stat counters and functions

2019-02-11 Thread Johannes Weiner
nconsistent names in both the percpu and > aggregated structs, so these are all cleaned up to be more coherent and > easy to understand. > > This commit contains code cleanup only: there are no logic changes. > > Signed-off-by: Chris Down > Cc: Andrew Morton > Cc: Michal Hocko

[PATCH] kernel: workqueue: clarify wq_worker_last_func() caller requirements

2019-02-06 Thread Johannes Weiner
This function can only be called safely from very specific scheduler contexts. Document those. Suggested-by: Andrew Morton Signed-off-by: Johannes Weiner --- kernel/workqueue.c | 10 ++ 1 file changed, 10 insertions(+) Andrew suggested including the explanations that came up during

Re: [PATCH v4] mm: Make memory.emin the baseline for utilisation determination

2019-02-06 Thread Johannes Weiner
, and > another cgroup without any emin. In both cgroups, set an elow larger > than 50% of physical RAM. The one with emin will have less page > scanning, as reclaim pressure is lower. > > Signed-off-by: Chris Down > Suggested-by: Roman Gushchin > Cc: Johannes Weiner > Cc:

[tip:x86/urgent] x86/resctrl: Avoid confusion over the new X86_RESCTRL config

2019-02-02 Thread tip-bot for Johannes Weiner
Commit-ID: e6d429313ea5c776d2e76b4494df69102e6b7115 Gitweb: https://git.kernel.org/tip/e6d429313ea5c776d2e76b4494df69102e6b7115 Author: Johannes Weiner AuthorDate: Tue, 29 Jan 2019 17:44:36 -0500 Committer: Borislav Petkov CommitDate: Sat, 2 Feb 2019 10:34:52 +0100 x86/resctrl: Avoid

Re: [PATCH 2/2] mm: Consider subtrees in memory.events

2019-02-01 Thread Johannes Weiner
On Fri, Feb 01, 2019 at 11:27:41AM +0100, Michal Hocko wrote: > On Thu 31-01-19 11:22:48, Johannes Weiner wrote: > > On Thu, Jan 31, 2019 at 09:58:08AM +0100, Michal Hocko wrote: > > > On Wed 30-01-19 16:31:31, Johannes Weiner wrote: > > > > On Wed, Jan 30, 2019 at

Re: [PATCH] mm: Throttle allocators when failing reclaim over memory.high

2019-02-01 Thread Johannes Weiner
On Fri, Feb 01, 2019 at 08:17:57AM +0100, Michal Hocko wrote: > On Thu 31-01-19 20:13:52, Chris Down wrote: > [...] > > The current situation goes against both the expectations of users of > > memory.high, and our intentions as cgroup v2 developers. In > > cgroup-v2.txt, we claim that we will

Re: [PATCH 2/2] mm: Consider subtrees in memory.events

2019-01-31 Thread Johannes Weiner
On Thu, Jan 31, 2019 at 09:58:08AM +0100, Michal Hocko wrote: > On Wed 30-01-19 16:31:31, Johannes Weiner wrote: > > On Wed, Jan 30, 2019 at 09:05:59PM +0100, Michal Hocko wrote: > [...] > > > I thought I have already mentioned an example. Say you have an observer > >

Re: [PATCH 2/2] mm: Consider subtrees in memory.events

2019-01-30 Thread Johannes Weiner
On Wed, Jan 30, 2019 at 09:05:59PM +0100, Michal Hocko wrote: > On Wed 30-01-19 14:23:45, Johannes Weiner wrote: > > On Mon, Jan 28, 2019 at 01:51:51PM +0100, Michal Hocko wrote: > > > On Fri 25-01-19 10:28:08, Tejun Heo wrote: > > > > On Fri, Jan 25, 2019 at 06:37:

Re: [PATCH] x86: avoid confusion over the new RESCTRL config prompt

2019-01-30 Thread Johannes Weiner
On Wed, Jan 30, 2019 at 06:19:33PM +0100, Borislav Petkov wrote: > On Wed, Jan 30, 2019 at 11:46:26AM -0500, Johannes Weiner wrote: > > This is still awefully close to the cgroup CPU resource > > controller. Since it's more hardware-specific, and the config symbol > > also h

Re: [PATCH 2/2] mm: Consider subtrees in memory.events

2019-01-30 Thread Johannes Weiner
On Wed, Jan 30, 2019 at 02:27:12PM -0500, Johannes Weiner wrote: > On Wed, Jan 30, 2019 at 11:11:44AM -0800, Shakeel Butt wrote: > > Hi Tejun, > > > > On Wed, Jan 30, 2019 at 9:07 AM Tejun Heo wrote: > > > > > > Hello, Michal. > > > > > &g

Re: [PATCH 2/2] mm: Consider subtrees in memory.events

2019-01-30 Thread Johannes Weiner
On Wed, Jan 30, 2019 at 11:11:44AM -0800, Shakeel Butt wrote: > Hi Tejun, > > On Wed, Jan 30, 2019 at 9:07 AM Tejun Heo wrote: > > > > Hello, Michal. > > > > On Wed, Jan 30, 2019 at 05:50:58PM +0100, Michal Hocko wrote: > > > > Yeah, cgroup.events and .stat files as some of the local stats would

Re: [PATCH 2/2] mm: Consider subtrees in memory.events

2019-01-30 Thread Johannes Weiner
On Mon, Jan 28, 2019 at 01:51:51PM +0100, Michal Hocko wrote: > On Fri 25-01-19 10:28:08, Tejun Heo wrote: > > On Fri, Jan 25, 2019 at 06:37:13PM +0100, Michal Hocko wrote: > > > Please note that I understand that this might be confusing with the rest > > > of the cgroup APIs but considering that

Re: [PATCH] x86: avoid confusion over the new RESCTRL config prompt

2019-01-30 Thread Johannes Weiner
On Wed, Jan 30, 2019 at 12:08:45AM +0100, Borislav Petkov wrote: > On Tue, Jan 29, 2019 at 05:52:18PM -0500, Johannes Weiner wrote: > > "Resource Control" is a very broad term for this CPU feature, and a > > term that is also associated with containers, cgroups etc.

[PATCH] psi: clarify the Kconfig text for the default-disable option

2019-01-29 Thread Johannes Weiner
-off-by: Johannes Weiner --- init/Kconfig | 11 +++ 1 file changed, 11 insertions(+) diff --git a/init/Kconfig b/init/Kconfig index 513fa544a134..ad3381e57402 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -512,6 +512,17 @@ config PSI_DEFAULT_DISABLED per default but can

Re: [RFC v2 PATCH] mm: vmscan: do not iterate all mem cgroups for global direct reclaim

2019-01-29 Thread Johannes Weiner
he cached iterator cookie could help to achieve the fairness more or > less. > > Cc: Johannes Weiner > Cc: Michal Hocko > Signed-off-by: Yang Shi Looks sane to me, thanks Yang. Acked-by: Johannes Weiner

[PATCH] x86: avoid confusion over the new RESCTRL config prompt

2019-01-29 Thread Johannes Weiner
"Resource Control" is a very broad term for this CPU feature, and a term that is also associated with containers, cgroups etc. This can easily cause confusion. Make the user prompt more specific. Signed-off-by: Johannes Weiner --- arch/x86/Kconfig | 4 ++-- 1 file changed, 2 insert

Re: [PATCH] mm: memcontrol: Expose THP events on a per-memcg basis

2019-01-29 Thread Johannes Weiner
ed to the page. It's a good idea to rework > that, but let's leave untangling THP allocation for a future patch. > > Signed-off-by: Chris Down > Cc: Andrew Morton > Cc: Johannes Weiner > Cc: Tejun Heo > Cc: Roman Gushchin > Cc: linux-kernel@vger.kernel.org > Cc: cgro...

Re: [PATCH v3 5/5] psi: introduce psi monitor

2019-01-29 Thread Johannes Weiner
cu(); > > + psi_trigger_destroy(t); > > +} > > +#endif /* CONFIG_PSI */ > > > > static int cgroup_file_open(struct kernfs_open_file *of) > > { > > @@ -4619,18 +4713,27 @@ static struct cftype cgroup_base_files[] = { > > .name = "io.pressure&

Re: [PATCH] psi: fix aggregation idle shut-off

2019-01-28 Thread Johannes Weiner
On Mon, Jan 28, 2019 at 02:03:36PM -0800, Andrew Morton wrote: > On Wed, 16 Jan 2019 14:35:01 -0500 Johannes Weiner wrote: > > +/** > > + * wq_worker_last_func - retrieve worker's last work function > > + * > > + * Determine the last function a worker ex

Re: [PATCH v3] oom, oom_reaper: do not enqueue same task twice

2019-01-28 Thread Johannes Weiner
Hi Tetsuo, On Sun, Jan 27, 2019 at 11:57:38PM +0900, Tetsuo Handa wrote: > From 9c9e935fc038342c48461aabca666f1b544e32b1 Mon Sep 17 00:00:00 2001 > From: Tetsuo Handa > Date: Sun, 27 Jan 2019 23:51:37 +0900 > Subject: [PATCH v3] oom, oom_reaper: do not enqueue same task twice > > Arkadiusz

Re: [PATCH] mm,slab,vmscan: accumulate gradual pressure on small slabs

2019-01-28 Thread Johannes Weiner
tively used entries from smaller caches get reclaimed less > aggressively, and need to be reloaded/recreated less often. > > Fixes: 4b85afbdacd2 ("mm: zero-seek shrinkers") > Fixes: 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number > of objects&qu

Re: [PATCH v3 5/5] psi: introduce psi monitor

2019-01-28 Thread Johannes Weiner
One thought on the v3 delta that I missed earlier: On Thu, Jan 24, 2019 at 01:15:18PM -0800, Suren Baghdasaryan wrote: > +/* > + * psi_update_work represents slowpath accounting part while psi_group_change > + * represents hotpath part. There are two potential races between them: > + * 1. Changes

Re: [PATCH v3 4/5] psi: rename psi fields in preparation for psi trigger addition

2019-01-28 Thread Johannes Weiner
asaryan Acked-by: Johannes Weiner

Re: [PATCH v3 3/5] psi: introduce state_mask to represent stalled psi states

2019-01-28 Thread Johannes Weiner
gt; > Signed-off-by: Suren Baghdasaryan Acked-by: Johannes Weiner

Re: + memcg-do-not-report-racy-no-eligible-oom-tasks.patch added to -mm tree

2019-01-25 Thread Johannes Weiner
On Fri, Jan 25, 2019 at 06:24:16PM +0100, Michal Hocko wrote: > On Fri 25-01-19 11:56:24, Johannes Weiner wrote: > > On Wed, Jan 09, 2019 at 11:03:06AM -0800, a...@linux-foundation.org wrote: > > > > > > The patch titled > > > Subject: memcg: do n

Re: + memcg-do-not-report-racy-no-eligible-oom-tasks.patch added to -mm tree

2019-01-25 Thread Johannes Weiner
close the race. > > [1] > http://lkml.kernel.org/r/bb2074c0-34fe-8c2c-1c7d-db71338f1...@i-love.sakura.ne.jp > Link: http://lkml.kernel.org/r/20190107143802.16847-3-mho...@kernel.org > Signed-off-by: Michal Hocko > Cc: Tetsuo Handa > Cc: Johannes Weiner > Signed-off-by: Andre

Re: [PATCH 2/2] mm: Extract memcg maxable seq_file logic to seq_show_memcg_tunable

2019-01-24 Thread Johannes Weiner
unction to do this > work. In combination with the previous patch adding mem_cgroup_from_seq, > this makes all of the seq_show feeder functions significantly more > simple. > > Signed-off-by: Chris Down > Cc: Andrew Morton > Cc: Johannes Weiner > Cc: Tejun Heo > Cc

Re: [PATCH 1/2] mm: Create mem_cgroup_from_seq

2019-01-24 Thread Johannes Weiner
quires manually getting the css, then getting the mem_cgroup > from the css. It's in enough places now that having mem_cgroup_from_seq > makes sense (and also makes the next patch a bit nicer). > > Signed-off-by: Chris Down > Cc: Andrew Morton > Cc: Johannes Weiner > Cc: Tejun He

Re: [PATCH 2/2] mm: Consider subtrees in memory.events

2019-01-24 Thread Johannes Weiner
On Thu, Jan 24, 2019 at 06:01:17PM +0100, Michal Hocko wrote: > On Thu 24-01-19 11:00:10, Johannes Weiner wrote: > [...] > > We cannot fully eliminate a risk for regression, but it strikes me as > > highly unlikely, given the extremely young age of cgroup2-based sys

Re: [PATCH] mm: Move maxable seq_file logic into a single place

2019-01-24 Thread Johannes Weiner
t; implements a generic way to do this work, avoiding fragmenting logic. > > Signed-off-by: Chris Down > Cc: Andrew Morton > Cc: Johannes Weiner > Cc: Tejun Heo > Cc: Roman Gushchin > Cc: linux-kernel@vger.kernel.org > Cc: cgro...@vger.kernel.org > Cc: li

Re: [PATCH 2/2] mm: Consider subtrees in memory.events

2019-01-24 Thread Johannes Weiner
On Thu, Jan 24, 2019 at 09:22:52AM +0100, Michal Hocko wrote: > On Wed 23-01-19 17:31:44, Chris Down wrote: > > Acked-by: Johannes Weiner > > btw. I do not see this patch posted anywhere yet it already comes with > an ack. Have I just missed a previous version? I reviewed

[PATCH] psi: fix aggregation idle shut-off

2019-01-16 Thread Johannes Weiner
: eb414681d5a0 ("psi: pressure stall information for CPU, memory, and IO") Reported-by: Suren Baghdasaryan Signed-off-by: Johannes Weiner --- kernel/sched/psi.c | 21 + kernel/workqueue.c | 23 +++ kernel/workqueue_internal.h | 6

Re: [PATCH v2 5/5] psi: introduce psi monitor

2019-01-16 Thread Johannes Weiner
On Wed, Jan 16, 2019 at 02:17:28PM -0500, Johannes Weiner wrote: > On Wed, Jan 16, 2019 at 09:39:13AM -0800, Suren Baghdasaryan wrote: > > On Wed, Jan 16, 2019 at 5:24 AM Peter Zijlstra wrote: > > > > > > On Mon, Jan 14, 2019 at 11:30:12AM -0800, Suren Baghdasarya

Re: [PATCH v2 5/5] psi: introduce psi monitor

2019-01-16 Thread Johannes Weiner
On Wed, Jan 16, 2019 at 09:39:13AM -0800, Suren Baghdasaryan wrote: > On Wed, Jan 16, 2019 at 5:24 AM Peter Zijlstra wrote: > > > > On Mon, Jan 14, 2019 at 11:30:12AM -0800, Suren Baghdasaryan wrote: > > > For memory ordering (which Johannes also pointed out) the critical point > > > is: > > > >

Re: [PATCH v2 5/5] psi: introduce psi monitor

2019-01-14 Thread Johannes Weiner
On Mon, Jan 14, 2019 at 11:30:12AM -0800, Suren Baghdasaryan wrote: > On Mon, Jan 14, 2019 at 2:22 AM Peter Zijlstra wrote: > > > > On Thu, Jan 10, 2019 at 02:07:18PM -0800, Suren Baghdasaryan wrote: > > > +/* > > > + * psi_update_work represents slowpath accounting part while > > > + *

Re: [RFC v3 PATCH 0/5] mm: memcontrol: do memory reclaim when offlining

2019-01-14 Thread Johannes Weiner
On Wed, Jan 09, 2019 at 05:47:41PM -0800, Yang Shi wrote: > On 1/9/19 2:51 PM, Johannes Weiner wrote: > > On Wed, Jan 09, 2019 at 02:09:20PM -0800, Yang Shi wrote: > > > On 1/9/19 1:23 PM, Johannes Weiner wrote: > > > > On Wed, Jan 09, 2019 at 12:36:11PM -0800,

Re: [PATCH v3] memcg: schedule high reclaim for remote memcgs on high_work

2019-01-11 Thread Johannes Weiner
Hi Shakeel, On Thu, Jan 10, 2019 at 09:44:32AM -0800, Shakeel Butt wrote: > If a memcg is over high limit, memory reclaim is scheduled to run on > return-to-userland. However it is assumed that the memcg is the current > process's memcg. With remote memcg charging for kmem or swapping in a >

Re: [RFC v3 PATCH 0/5] mm: memcontrol: do memory reclaim when offlining

2019-01-09 Thread Johannes Weiner
On Wed, Jan 09, 2019 at 02:09:20PM -0800, Yang Shi wrote: > On 1/9/19 1:23 PM, Johannes Weiner wrote: > > On Wed, Jan 09, 2019 at 12:36:11PM -0800, Yang Shi wrote: > > > As I mentioned above, if we know some page caches from some memcgs > > > are referenced one-off an

Re: [RFC v3 PATCH 0/5] mm: memcontrol: do memory reclaim when offlining

2019-01-09 Thread Johannes Weiner
On Wed, Jan 09, 2019 at 12:36:11PM -0800, Yang Shi wrote: > As I mentioned above, if we know some page caches from some memcgs > are referenced one-off and unlikely shared, why just keep them > around to increase memory pressure? It's just not clear to me that your scenarios are generic enough to

Re: [RFC v3 PATCH 0/5] mm: memcontrol: do memory reclaim when offlining

2019-01-09 Thread Johannes Weiner
On Thu, Jan 10, 2019 at 03:14:40AM +0800, Yang Shi wrote: > > We have some usecases which create and remove memcgs very frequently, > and the tasks in the memcg may just access the files which are unlikely > accessed by anyone else. So, we prefer force_empty the memcg before > rmdir'ing it to

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

2019-01-09 Thread Johannes Weiner
On Wed, Jan 09, 2019 at 09:44:28AM -0800, Shakeel Butt wrote: > Hi Johannes, > > On Wed, Jan 9, 2019 at 8:45 AM Johannes Weiner wrote: > > > > On Wed, Jan 09, 2019 at 03:20:18PM +0300, Kirill Tkhai wrote: > > > On nodes without memory overcommit, it's common a situa

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

2019-01-09 Thread Johannes Weiner
On Wed, Jan 09, 2019 at 03:20:18PM +0300, Kirill Tkhai wrote: > On nodes without memory overcommit, it's common a situation, > when memcg exceeds its limit and pages from pagecache are > shrinked on reclaim, while node has a lot of free memory. > Further access to the pages requires real device

Re: possible deadlock in __wake_up_common_lock

2019-01-07 Thread Johannes Weiner
On Mon, Jan 07, 2019 at 10:52:17AM +0100, Peter Zijlstra wrote: > On Wed, Jan 02, 2019 at 01:51:01PM +0100, Vlastimil Babka wrote: > > > -> #3 (>lock){-.-.}: > > > __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 > > > [inline] > > > _raw_spin_lock_irqsave+0x99/0xd0

Re: [PATCH 6/6] psi: introduce psi monitor

2018-12-18 Thread Johannes Weiner
On Tue, Dec 18, 2018 at 11:46:22AM +0100, Peter Zijlstra wrote: > On Mon, Dec 17, 2018 at 05:21:05PM -0800, Suren Baghdasaryan wrote: > > On Mon, Dec 17, 2018 at 8:22 AM Peter Zijlstra wrote: > > > > How well has this thing been fuzzed? Custom string parser, yay! > > > > Honestly, not much.

Re: [PATCH v3] mm, memcg: fix reclaim deadlock with writeback

2018-12-13 Thread Johannes Weiner
hidden __GFP_ACCOUNT | GFP_KERNEL allocations > from under a fs page locked but they should be really rare. I am not > aware of a better solution unfortunately. > > Reported-and-Debugged-by: Liu Bo > Cc: stable > Fixes: c3b94f44fcb0 ("memcg: fu

Re: [PATCH 3/4] filemap: drop the mmap_sem for all blocking operations

2018-12-05 Thread Johannes Weiner
t the fault handler makes things a lot simpler than the -EAGAIN and wait_on_page_locked dance from earlier versions. Nice. Acked-by: Johannes Weiner

Re: [PATCH 3/4] filemap: drop the mmap_sem for all blocking operations

2018-12-05 Thread Johannes Weiner
t the fault handler makes things a lot simpler than the -EAGAIN and wait_on_page_locked dance from earlier versions. Nice. Acked-by: Johannes Weiner

Re: [PATCH 2/4] filemap: kill page_cache_read usage in filemap_fault

2018-12-05 Thread Johannes Weiner
cache case significantly. > > Signed-off-by: Josef Bacik That's a great simplification. Looks correct to me. Acked-by: Johannes Weiner

Re: [PATCH 2/4] filemap: kill page_cache_read usage in filemap_fault

2018-12-05 Thread Johannes Weiner
cache case significantly. > > Signed-off-by: Josef Bacik That's a great simplification. Looks correct to me. Acked-by: Johannes Weiner

Re: [PATCH] psi: fix reference to kernel commandline enable

2018-12-03 Thread Johannes Weiner
ling/enabling easier for vendor kernels") > Signed-off-by: Baruch Siach Doh, thanks Baruch. Acked-by: Johannes Weiner > --- > init/Kconfig | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/init/Kconfig b/init/Kconfig > index cf5b5a0dcbc2..ed93525

Re: [PATCH] psi: fix reference to kernel commandline enable

2018-12-03 Thread Johannes Weiner
ling/enabling easier for vendor kernels") > Signed-off-by: Baruch Siach Doh, thanks Baruch. Acked-by: Johannes Weiner > --- > init/Kconfig | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/init/Kconfig b/init/Kconfig > index cf5b5a0dcbc2..ed93525

Re: Hackbench pipes regression bisected to PSI

2018-11-27 Thread Johannes Weiner
On Tue, Nov 27, 2018 at 11:46:17AM -0500, Johannes Weiner wrote: > From 347b69a52d1ec7e71df1108cbc5703d6dd0616ba Mon Sep 17 00:00:00 2001 > From: Johannes Weiner > Date: Mon, 26 Nov 2018 09:39:23 -0500 > Subject: [PATCH] psi: make disabling/enabling easier for vendor kernels >

Re: Hackbench pipes regression bisected to PSI

2018-11-27 Thread Johannes Weiner
On Tue, Nov 27, 2018 at 11:46:17AM -0500, Johannes Weiner wrote: > From 347b69a52d1ec7e71df1108cbc5703d6dd0616ba Mon Sep 17 00:00:00 2001 > From: Johannes Weiner > Date: Mon, 26 Nov 2018 09:39:23 -0500 > Subject: [PATCH] psi: make disabling/enabling easier for vendor kernels >

Re: Hackbench pipes regression bisected to PSI

2018-11-27 Thread Johannes Weiner
ed-by: Mel Gorman Thanks for testing it. Let's add these results to the changelog: --- >From 347b69a52d1ec7e71df1108cbc5703d6dd0616ba Mon Sep 17 00:00:00 2001 From: Johannes Weiner Date: Mon, 26 Nov 2018 09:39:23 -0500 Subject: [PATCH] psi: make disabling/enabling easier for vendor kernels Mel Gorman reports a

Re: Hackbench pipes regression bisected to PSI

2018-11-27 Thread Johannes Weiner
ed-by: Mel Gorman Thanks for testing it. Let's add these results to the changelog: --- >From 347b69a52d1ec7e71df1108cbc5703d6dd0616ba Mon Sep 17 00:00:00 2001 From: Johannes Weiner Date: Mon, 26 Nov 2018 09:39:23 -0500 Subject: [PATCH] psi: make disabling/enabling easier for vendor kernels Mel Gorman reports a

Re: Hackbench pipes regression bisected to PSI

2018-11-26 Thread Johannes Weiner
On Mon, Nov 26, 2018 at 04:54:47PM +, Mel Gorman wrote: > On Mon, Nov 26, 2018 at 11:07:24AM -0500, Johannes Weiner wrote: > > @@ -509,6 +509,15 @@ config PSI > > > > Say N if unsure. > > > > +config PSI_DEFAULT_DISABLED > > + bool "Req

Re: Hackbench pipes regression bisected to PSI

2018-11-26 Thread Johannes Weiner
On Mon, Nov 26, 2018 at 04:54:47PM +, Mel Gorman wrote: > On Mon, Nov 26, 2018 at 11:07:24AM -0500, Johannes Weiner wrote: > > @@ -509,6 +509,15 @@ config PSI > > > > Say N if unsure. > > > > +config PSI_DEFAULT_DISABLED > > + bool "Req

Re: Hackbench pipes regression bisected to PSI

2018-11-26 Thread Johannes Weiner
ast good commit: eb414681d5a07d28d2ff90dc05f69ec6b232ebd2 > First bad commit: 2ce7135adc9ad081aa3c49744144376ac74fea60 > From 2ce7135adc9ad081aa3c49744144376ac74fea60 Mon Sep 17 00:00:00 2001 > From: Johannes Weiner > Date: Fri, 26 Oct 2018 15:06:31 -0700 > Subject: [PATCH

Re: Hackbench pipes regression bisected to PSI

2018-11-26 Thread Johannes Weiner
ast good commit: eb414681d5a07d28d2ff90dc05f69ec6b232ebd2 > First bad commit: 2ce7135adc9ad081aa3c49744144376ac74fea60 > From 2ce7135adc9ad081aa3c49744144376ac74fea60 Mon Sep 17 00:00:00 2001 > From: Johannes Weiner > Date: Fri, 26 Oct 2018 15:06:31 -0700 > Subject: [PATCH

Re: [PATCH] psi: simplify cgroup_move_task

2018-11-06 Thread Johannes Weiner
; > Warning was: > > kernel/sched/psi.c: In function ‘cgroup_move_task’: > kernel/sched/psi.c:639:13: warning: ‘rq’ may be used uninitialized in this > function [-Wmaybe-uninitialized] > > Fixes: 2ce7135adc9ad ("psi: cgroup support") > Cc: Johannes Weiner > S

Re: [PATCH] psi: simplify cgroup_move_task

2018-11-06 Thread Johannes Weiner
; > Warning was: > > kernel/sched/psi.c: In function ‘cgroup_move_task’: > kernel/sched/psi.c:639:13: warning: ‘rq’ may be used uninitialized in this > function [-Wmaybe-uninitialized] > > Fixes: 2ce7135adc9ad ("psi: cgroup support") > Cc: Johannes Weiner > S

Re: [RFC PATCH 2/2] memcg: do not report racy no-eligible OOM tasks

2018-10-26 Thread Johannes Weiner
On Mon, Oct 22, 2018 at 09:13:23AM +0200, Michal Hocko wrote: > From: Michal Hocko > > Tetsuo has reported [1] that a single process group memcg might easily > swamp the log with no-eligible oom victim reports due to race between > the memcg charge and oom_reaper > > Thread 1

Re: [RFC PATCH 2/2] memcg: do not report racy no-eligible OOM tasks

2018-10-26 Thread Johannes Weiner
On Mon, Oct 22, 2018 at 09:13:23AM +0200, Michal Hocko wrote: > From: Michal Hocko > > Tetsuo has reported [1] that a single process group memcg might easily > swamp the log with no-eligible oom victim reports due to race between > the memcg charge and oom_reaper > > Thread 1

Re: [PATCH 0/9] psi: pressure stall information for CPU, memory, and IO v4

2018-10-23 Thread Johannes Weiner
On Thu, Oct 18, 2018 at 07:07:10PM -0700, Andrew Morton wrote: > On Tue, 28 Aug 2018 13:22:49 -0400 Johannes Weiner wrote: > > > This version 4 of the PSI series incorporates feedback from Peter and > > fixes two races in the lockless aggregator that Suren found in his >

Re: [PATCH 0/9] psi: pressure stall information for CPU, memory, and IO v4

2018-10-23 Thread Johannes Weiner
On Thu, Oct 18, 2018 at 07:07:10PM -0700, Andrew Morton wrote: > On Tue, 28 Aug 2018 13:22:49 -0400 Johannes Weiner wrote: > > > This version 4 of the PSI series incorporates feedback from Peter and > > fixes two races in the lockless aggregator that Suren found in his >

Re: [RFC PATCH] memcg, oom: throttle dump_header for memcg ooms without eligible tasks

2018-10-13 Thread Johannes Weiner
On Sat, Oct 13, 2018 at 08:09:30PM +0900, Tetsuo Handa wrote: > -- Michal's patch -- > > 73133 lines (5.79MB) of kernel messages per one run > > [root@ccsecurity ~]# time ./a.out > > real3m44.389s > user0m0.000s > sys 3m42.334s > > [root@ccsecurity ~]# time ./a.out

Re: [RFC PATCH] memcg, oom: throttle dump_header for memcg ooms without eligible tasks

2018-10-13 Thread Johannes Weiner
On Sat, Oct 13, 2018 at 08:09:30PM +0900, Tetsuo Handa wrote: > -- Michal's patch -- > > 73133 lines (5.79MB) of kernel messages per one run > > [root@ccsecurity ~]# time ./a.out > > real3m44.389s > user0m0.000s > sys 3m42.334s > > [root@ccsecurity ~]# time ./a.out

Re: [RFC PATCH] memcg, oom: throttle dump_header for memcg ooms without eligible tasks

2018-10-12 Thread Johannes Weiner
; information to go on already. > > > > Yeah. Unless we have a storm coming from many different cgroups in > > parallel. But even then we have the allocation context for each OOM so > > we are not losing everything. Should we ever tune this, it can be done > > later with

Re: [RFC PATCH] memcg, oom: throttle dump_header for memcg ooms without eligible tasks

2018-10-12 Thread Johannes Weiner
; information to go on already. > > > > Yeah. Unless we have a storm coming from many different cgroups in > > parallel. But even then we have the allocation context for each OOM so > > we are not losing everything. Should we ever tune this, it can be done > > later with

Re: [RFC PATCH] memcg, oom: throttle dump_header for memcg ooms without eligible tasks

2018-10-12 Thread Johannes Weiner
.ne.jp > Cc: rient...@google.com > Cc: yan...@alibaba-inc.com > Signed-off-by: Michal Hocko So not more than 10 dumps in each 5s interval. That looks reasonable to me. By the time it starts dropping data you have more than enough information to go on already. Acked-by: Johannes Weiner

Re: [RFC PATCH] memcg, oom: throttle dump_header for memcg ooms without eligible tasks

2018-10-12 Thread Johannes Weiner
.ne.jp > Cc: rient...@google.com > Cc: yan...@alibaba-inc.com > Signed-off-by: Michal Hocko So not more than 10 dumps in each 5s interval. That looks reasonable to me. By the time it starts dropping data you have more than enough information to go on already. Acked-by: Johannes Weiner

Re: [PATCH 4/4] mm: zero-seek shrinkers

2018-10-10 Thread Johannes Weiner
On Wed, Oct 10, 2018 at 01:03:50AM +, Rik van Riel wrote: > On Tue, 2018-10-09 at 14:47 -0400, Johannes Weiner wrote: > > > These workloads also deal with tens of thousands of open files and > > use > > /proc for introspection, which ends up growing the proc_inode_c

Re: [PATCH 4/4] mm: zero-seek shrinkers

2018-10-10 Thread Johannes Weiner
On Wed, Oct 10, 2018 at 01:03:50AM +, Rik van Riel wrote: > On Tue, 2018-10-09 at 14:47 -0400, Johannes Weiner wrote: > > > These workloads also deal with tens of thousands of open files and > > use > > /proc for introspection, which ends up growing the proc_inode_c

Re: [PATCH 3/4] mm: workingset: add vmstat counter for shadow nodes

2018-10-10 Thread Johannes Weiner
On Tue, Oct 09, 2018 at 03:08:45PM -0700, Andrew Morton wrote: > On Tue, 9 Oct 2018 14:47:32 -0400 Johannes Weiner wrote: > > > --- a/mm/workingset.c > > +++ b/mm/workingset.c > > @@ -378,11 +378,17 @@ void workingset_update_node(struct xa_node *node) > >

Re: [PATCH 3/4] mm: workingset: add vmstat counter for shadow nodes

2018-10-10 Thread Johannes Weiner
On Tue, Oct 09, 2018 at 03:08:45PM -0700, Andrew Morton wrote: > On Tue, 9 Oct 2018 14:47:32 -0400 Johannes Weiner wrote: > > > --- a/mm/workingset.c > > +++ b/mm/workingset.c > > @@ -378,11 +378,17 @@ void workingset_update_node(struct xa_node *node) > >

Re: [PATCH 3/4] mm: workingset: add vmstat counter for shadow nodes

2018-10-10 Thread Johannes Weiner
On Tue, Oct 09, 2018 at 03:04:01PM -0700, Andrew Morton wrote: > On Tue, 9 Oct 2018 14:47:32 -0400 Johannes Weiner wrote: > > > Make it easier to catch bugs in the shadow node shrinker by adding a > > counter for the shadow nodes in circulation. > > > >

Re: [PATCH 3/4] mm: workingset: add vmstat counter for shadow nodes

2018-10-10 Thread Johannes Weiner
On Tue, Oct 09, 2018 at 03:04:01PM -0700, Andrew Morton wrote: > On Tue, 9 Oct 2018 14:47:32 -0400 Johannes Weiner wrote: > > > Make it easier to catch bugs in the shadow node shrinker by adding a > > counter for the shadow nodes in circulation. > > > >

[PATCH 1/4] mm: workingset: don't drop refault information prematurely fix

2018-10-09 Thread Johannes Weiner
cgroup size. Signed-off-by: Johannes Weiner --- mm/workingset.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/workingset.c b/mm/workingset.c index 1d111913929d..e5c70bc94077 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -418,9 +418,15 @@ static

[PATCH 1/4] mm: workingset: don't drop refault information prematurely fix

2018-10-09 Thread Johannes Weiner
cgroup size. Signed-off-by: Johannes Weiner --- mm/workingset.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/workingset.c b/mm/workingset.c index 1d111913929d..e5c70bc94077 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -418,9 +418,15 @@ static

[PATCH 3/4] mm: workingset: add vmstat counter for shadow nodes

2018-10-09 Thread Johannes Weiner
Make it easier to catch bugs in the shadow node shrinker by adding a counter for the shadow nodes in circulation. Signed-off-by: Johannes Weiner --- include/linux/mmzone.h | 1 + mm/vmstat.c| 1 + mm/workingset.c| 12 ++-- 3 files changed, 12 insertions(+), 2

[PATCH 3/4] mm: workingset: add vmstat counter for shadow nodes

2018-10-09 Thread Johannes Weiner
Make it easier to catch bugs in the shadow node shrinker by adding a counter for the shadow nodes in circulation. Signed-off-by: Johannes Weiner --- include/linux/mmzone.h | 1 + mm/vmstat.c| 1 + mm/workingset.c| 12 ++-- 3 files changed, 12 insertions(+), 2

[PATCH 2/4] mm: workingset: use cheaper __inc_lruvec_state in irqsafe node reclaim

2018-10-09 Thread Johannes Weiner
No need to use the preemption-safe lruvec state function inside the reclaim region that has irqs disabled. Signed-off-by: Johannes Weiner --- mm/workingset.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/workingset.c b/mm/workingset.c index e5c70bc94077..f564aaa6b71d

[PATCH 4/4] mm: zero-seek shrinkers

2018-10-09 Thread Johannes Weiner
zero-seek setting. Reported-by: Domas Mituzas Signed-off-by: Johannes Weiner --- fs/kernfs/mount.c | 3 +++ fs/proc/root.c| 3 +++ mm/vmscan.c | 15 --- mm/workingset.c | 2 +- 4 files changed, 19 insertions(+), 4 deletions(-) diff --git a/fs/kernfs/mount.c b/fs/kern

[PATCH 2/4] mm: workingset: use cheaper __inc_lruvec_state in irqsafe node reclaim

2018-10-09 Thread Johannes Weiner
No need to use the preemption-safe lruvec state function inside the reclaim region that has irqs disabled. Signed-off-by: Johannes Weiner --- mm/workingset.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/workingset.c b/mm/workingset.c index e5c70bc94077..f564aaa6b71d

[PATCH 4/4] mm: zero-seek shrinkers

2018-10-09 Thread Johannes Weiner
zero-seek setting. Reported-by: Domas Mituzas Signed-off-by: Johannes Weiner --- fs/kernfs/mount.c | 3 +++ fs/proc/root.c| 3 +++ mm/vmscan.c | 15 --- mm/workingset.c | 2 +- 4 files changed, 19 insertions(+), 4 deletions(-) diff --git a/fs/kernfs/mount.c b/fs/kern

<    2   3   4   5   6   7   8   9   10   11   >