Re: [PATCH] Revert "mm: rename _count, field of the struct page, to _refcount"

2016-06-16 Thread Michal Hocko
justified. struct page layout as some others that such a tool might depend on has changes several times in the past so I fail to see how is it any different this time. struct page is nothing the userspace should depend on. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Revert "mm: rename _count, field of the struct page, to _refcount"

2016-06-16 Thread Michal Hocko
On Thu 16-06-16 13:22:27, Vitaly Kuznetsov wrote: > Michal Hocko <mho...@kernel.org> writes: > > > On Thu 16-06-16 12:30:16, Vitaly Kuznetsov wrote: > >> Christoph Hellwig <h...@infradead.org> writes: > >> > >> > On Thu, Jun 16, 2016 at 11

Re: [PATCH 0037/1529] Fix typo

2016-05-23 Thread Michal Hocko
ugetlb" and "Private_Hugetlb" show the amounts of memory backed by > hugetlbfs page which is *not* counted in "RSS" or "PSS" field for historical > reasons. And these are not included in {Shared,Private}_{Clean,Dirty} field. > "Swap" sh

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-15 Thread Michal Hocko
On Mon 15-08-16 09:00:04, Robert Foss wrote: > > > On 2016-08-14 05:04 AM, Michal Hocko wrote: > > On Fri 12-08-16 18:04:19, robert.f...@collabora.com wrote: > > > From: Robert Foss <robert.f...@collabora.com> > > > > > > This series imp

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-16 Thread Michal Hocko
On Mon 15-08-16 12:25:10, Robert Foss wrote: > > > On 2016-08-15 09:42 AM, Michal Hocko wrote: [...] > > The use case is to speed up monitoring of > > memory consumption in environments where RSS isn't precise. > > > > For example Chrome tends to many proc

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-17 Thread Michal Hocko
On Wed 17-08-16 11:31:25, Jann Horn wrote: > On Wed, Aug 17, 2016 at 10:22:00AM +0200, Michal Hocko wrote: > > On Tue 16-08-16 12:46:51, Robert Foss wrote: > > [...] > > > $ /usr/bin/time -v -p zsh -c "repeat 25 { awk '/^Rss/{rss+=\$2} > > > /^Pss/{pss+=\$2

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-18 Thread Michal Hocko
On Thu 18-08-16 10:47:57, Sonny Rao wrote: > On Thu, Aug 18, 2016 at 12:44 AM, Michal Hocko <mho...@kernel.org> wrote: > > On Wed 17-08-16 11:57:56, Sonny Rao wrote: [...] > >> 2) User space OOM handling -- we'd rather do a more graceful shutdown > >> than let

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-22 Thread Michal Hocko
t; > we want to address it in a generic way as much as possible. > > Sure. What solution do you think as generic way? either optimize seq_printf or replace it with something faster. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-22 Thread Michal Hocko
On Fri 19-08-16 10:57:48, Sonny Rao wrote: > On Fri, Aug 19, 2016 at 12:59 AM, Michal Hocko <mho...@kernel.org> wrote: > > On Thu 18-08-16 23:43:39, Sonny Rao wrote: > >> On Thu, Aug 18, 2016 at 11:01 AM, Michal Hocko <mho...@kernel.org> wrote: > >> >

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-19 Thread Michal Hocko
On Thu 18-08-16 23:43:39, Sonny Rao wrote: > On Thu, Aug 18, 2016 at 11:01 AM, Michal Hocko <mho...@kernel.org> wrote: > > On Thu 18-08-16 10:47:57, Sonny Rao wrote: > >> On Thu, Aug 18, 2016 at 12:44 AM, Michal Hocko <mho...@kernel.org> wrote: > >> >

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-19 Thread Michal Hocko
On Fri 19-08-16 11:26:34, Minchan Kim wrote: > Hi Michal, > > On Thu, Aug 18, 2016 at 08:01:04PM +0200, Michal Hocko wrote: > > On Thu 18-08-16 10:47:57, Sonny Rao wrote: > > > On Thu, Aug 18, 2016 at 12:44 AM, Michal Hocko <mho...@kernel.org> wrote: > > >

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-30 Thread Michal Hocko
ound arguments so far. Everything was just "we know what we are doing in our environment so we know those resouces and therefore those numbers make sense to us". But with all due respect this is not a reason to add a user visible API into the kernel. -- Michal Hocko SUSE Labs -- To u

Re: [PATCH v5 0/3] mm, proc: Implement /proc//totmaps

2016-09-12 Thread Michal Hocko
On Mon 12-09-16 08:31:36, Sonny Rao wrote: > On Mon, Sep 12, 2016 at 5:02 AM, Michal Hocko <mho...@kernel.org> wrote: > > On Mon 05-09-16 16:14:06, robert.f...@collabora.com wrote: > >> From: Robert Foss <robert.f...@collabora.com> > >> > >> Th

Re: [PATCH v5 0/3] mm, proc: Implement /proc//totmaps

2016-09-14 Thread Michal Hocko
On Tue 13-09-16 13:27:39, Sonny Rao wrote: > On Tue, Sep 13, 2016 at 12:12 AM, Michal Hocko <mho...@kernel.org> wrote: > > On Mon 12-09-16 10:28:53, Sonny Rao wrote: > >> On Mon, Sep 12, 2016 at 10:15 AM, Michal Hocko <mho...@kernel.org> wrote: > >> >

Re: [PATCH v5 0/3] mm, proc: Implement /proc//totmaps

2016-09-13 Thread Michal Hocko
On Mon 12-09-16 10:28:53, Sonny Rao wrote: > On Mon, Sep 12, 2016 at 10:15 AM, Michal Hocko <mho...@kernel.org> wrote: > > On Mon 12-09-16 08:31:36, Sonny Rao wrote: [...] > >> but how about the other fields like Swap, Private_Dirty and > >> Private_Shared? >

Re: utime accounting regression since 4.6 (was: Re: [PACTH v2 0/3] Implement /proc//totmaps)

2016-09-30 Thread Michal Hocko
:33 +0200, Michal Hocko wrote: [...] > > OK, so it seems I found it. I was quite lucky because > > account_user_time > > is not all that popular function and there were basically no changes > > besides Riks ff9a9b4c4334 ("sched, time: Switch > > VIRT_CPU_A

Re: [PATCH v5 0/3] mm, proc: Implement /proc//totmaps

2016-09-19 Thread Michal Hocko
On Mon 19-09-16 11:16:31, Robert Foss wrote: > On 2016-09-14 05:12 AM, Michal Hocko wrote: > > On Tue 13-09-16 13:27:39, Sonny Rao wrote: [...] > > > Given that smaps > > > doesn't provide this in a straightforward way, what do you think is > > > the r

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-17 Thread Michal Hocko
I am still very skeptical about a dedicated proc file which accomplishes what userspace can done in a trivial way. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-18 Thread Michal Hocko
On Wed 17-08-16 11:57:56, Sonny Rao wrote: > On Wed, Aug 17, 2016 at 6:03 AM, Michal Hocko <mho...@kernel.org> wrote: > > On Wed 17-08-16 11:31:25, Jann Horn wrote: [...] > >> That's at least 30.43% + 9.12% + 7.66% = 47.21% of the task's kernel > >> time spent on e

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-29 Thread Michal Hocko
[Sorry for a late reply, I was busy with other stuff] On Mon 22-08-16 15:44:53, Sonny Rao wrote: > On Mon, Aug 22, 2016 at 12:54 AM, Michal Hocko <mho...@kernel.org> wrote: [...] > But what about the private_clean and private_dirty? Surely > those are more generally useful

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-22 Thread Michal Hocko
On Mon 22-08-16 19:29:36, Michal Hocko wrote: > On Mon 22-08-16 18:45:54, Michal Hocko wrote: > [...] > > I have no idea why those numbers are so different on my laptop > > yet. It surely looks suspicious. I will try to debug this further > > tomorrow. > > Hmm, s

Re: [PACTH v2 0/3] Implement /proc//totmaps

2016-08-23 Thread Michal Hocko
On Mon 22-08-16 19:47:09, Michal Hocko wrote: > On Mon 22-08-16 19:29:36, Michal Hocko wrote: > > On Mon 22-08-16 18:45:54, Michal Hocko wrote: > > [...] > > > I have no idea why those numbers are so different on my laptop > > > yet. It surely looks suspicious.

utime accounting regression since 4.6 (was: Re: [PACTH v2 0/3] Implement /proc//totmaps)

2016-08-23 Thread Michal Hocko
On Tue 23-08-16 10:26:03, Michal Hocko wrote: > On Mon 22-08-16 19:47:09, Michal Hocko wrote: > > On Mon 22-08-16 19:29:36, Michal Hocko wrote: > > > On Mon 22-08-16 18:45:54, Michal Hocko wrote: > > > [...] > > > > I have no idea why those numbers ar

Re: [v4 2/4] mm, oom: cgroup-aware OOM killer

2017-08-02 Thread Michal Hocko
On Tue 01-08-17 19:13:52, Roman Gushchin wrote: > On Tue, Aug 01, 2017 at 07:03:03PM +0200, Michal Hocko wrote: > > On Tue 01-08-17 16:25:48, Roman Gushchin wrote: > > > On Tue, Aug 01, 2017 at 04:54:35PM +0200, Michal Hocko wrote: > > [...] > > > > I

Re: [v4 2/4] mm, oom: cgroup-aware OOM killer

2017-08-03 Thread Michal Hocko
On Thu 03-08-17 13:47:51, Roman Gushchin wrote: > On Wed, Aug 02, 2017 at 09:29:01AM +0200, Michal Hocko wrote: > > On Tue 01-08-17 19:13:52, Roman Gushchin wrote: > > > On Tue, Aug 01, 2017 at 07:03:03PM +0200, Michal Hocko wrote: > > > > On Tue 01-08-17

Re: [RFC v5 00/38] powerpc: Memory Protection Keys

2017-07-12 Thread Michal Hocko
On Tue 11-07-17 12:32:57, Ram Pai wrote: > On Tue, Jul 11, 2017 at 04:52:46PM +0200, Michal Hocko wrote: > > On Wed 05-07-17 14:21:37, Ram Pai wrote: > > > Memory protection keys enable applications to protect its > > > address space from inadvertent access or c

Re: [RFC v5 00/38] powerpc: Memory Protection Keys

2017-07-12 Thread Michal Hocko
On Wed 12-07-17 09:23:37, Michal Hocko wrote: > On Tue 11-07-17 12:32:57, Ram Pai wrote: [...] > > Ideally the MMU looks at the PTE for keys, in order to enforce > > protection. This is the case with x86 and is the case with power9 Radix > > page table. Hence the keys

Re: [RFC v5 00/38] powerpc: Memory Protection Keys

2017-07-11 Thread Michal Hocko
o you need to store anything in the pte? My understanding of PKEYs is that the setup and teardown should be very cheap and so no page tables have to updated. Or do I just misunderstand what you wrote here? -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linu

Re: [RFC v5 00/38] powerpc: Memory Protection Keys

2017-07-13 Thread Michal Hocko
On Thu 13-07-17 08:53:52, Benjamin Herrenschmidt wrote: > On Wed, 2017-07-12 at 09:23 +0200, Michal Hocko wrote: > > > > > > > > Ideally the MMU looks at the PTE for keys, in order to enforce > > > protection. This is the case with x86 and is the case with

Re: [v4 1/4] mm, oom: refactor the TIF_MEMDIE usage

2017-07-26 Thread Michal Hocko
ithreaded process can still deplete the reserves completely). Is there really any reason to not go with the existing patch I've pointed to the last time around? You didn't seem to have any objects back then. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsu

Re: [v4 1/4] mm, oom: refactor the TIF_MEMDIE usage

2017-07-26 Thread Michal Hocko
On Wed 26-07-17 15:06:07, Roman Gushchin wrote: > On Wed, Jul 26, 2017 at 03:56:22PM +0200, Michal Hocko wrote: > > On Wed 26-07-17 14:27:15, Roman Gushchin wrote: > > [...] > > > @@ -656,13 +658,24 @@ static void mark_oom_victim(struct task_struct *tsk) > > >

Re: [v4 1/4] mm, oom: refactor the TIF_MEMDIE usage

2017-07-26 Thread Michal Hocko
On Wed 26-07-17 16:24:34, Michal Hocko wrote: [...] > Or if you prefer I can post it separately? I've just tried to rebase relevant parts on top of the current mmotm tree and it needs some non-trivial updates. Would you mind if I post those patches with you on CC? I really think that we should

Re: [v3 5/6] mm, oom: don't mark all oom victims tasks with TIF_MEMDIE

2017-06-30 Thread Michal Hocko
On Thu 29-06-17 14:45:13, Roman Gushchin wrote: > On Thu, Jun 29, 2017 at 10:53:57AM +0200, Michal Hocko wrote: > > On Wed 21-06-17 22:19:15, Roman Gushchin wrote: > > > We want to limit the number of tasks which are having an access > > > to the memory reserves. To ensu

Re: [v3 5/6] mm, oom: don't mark all oom victims tasks with TIF_MEMDIE

2017-06-29 Thread Michal Hocko
the reduced memory reserves access to oom victims I was suggesting earlier [1]? [1] http://lkml.kernel.org/r/http://lkml.kernel.org/r/1472723464-22866-2-git-send-email-mho...@kernel.org -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc&quo

Re: [v3 1/6] mm, oom: use oom_victims counter to synchronize oom victim selection

2017-06-29 Thread Michal Hocko
in the oom hierarchy with oom victims which are alive and not MMF_OOM_SKIP, you should abort the scanning. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v4 2/4] mm, oom: cgroup-aware OOM killer

2017-08-01 Thread Michal Hocko
how it is implemented. I was hoping for something much simpler which would hook into oom_evaluate_task. If a task belongs to a memcg with kill-all flag then we would update the cumulative memcg badness (more specifically the badness of the topmost parent with kill-all flag). Memcg will then compete w

Re: [v4 2/4] mm, oom: cgroup-aware OOM killer

2017-08-01 Thread Michal Hocko
On Tue 01-08-17 16:25:48, Roman Gushchin wrote: > On Tue, Aug 01, 2017 at 04:54:35PM +0200, Michal Hocko wrote: [...] > > I would reap out the oom_kill_process into a separate patch. > > It was a separate patch, I've merged it based on Vladimir's feedback. > No problems, I

Re: [PATCH] mm, docs: update memory.stat description with workingset* entries

2017-05-16 Thread Michal Hocko
- workingset_nodereclaim. > > This commit adds a corresponding description to the cgroup v2 docs. > > Signed-off-by: Roman Gushchin <g...@fb.com> > Cc: Johannes Weiner <han...@cmpxchg.org> > Cc: Michal Hocko <mho...@kernel.org> > Cc: Vladimir Davydov <vdavydov

Re: [PATCH v2] mm: Allow slab_nomerge to be set at build time

2017-06-26 Thread Michal Hocko
On Fri 23-06-17 12:20:25, Kees Cook wrote: > On Fri, Jun 23, 2017 at 7:06 AM, Michal Hocko <mho...@kernel.org> wrote: > > On Tue 20-06-17 16:09:11, Kees Cook wrote: > >> Some hardened environments want to build kernels with slab_nomerge > >> already set (so that

Re: [PATCH v2] mm: Allow slab_nomerge to be set at build time

2017-06-23 Thread Michal Hocko
skeptics.) > */ > -static int slab_nomerge; > +static bool slab_nomerge = !IS_ENABLED(CONFIG_SLAB_MERGE_DEFAULT); > > static int __init setup_slab_nomerge(char *str) > { > - slab_nomerge = 1; > + slab_nomerge = true; > return 1; > } > > -- > 2.7.4 &

Re: [PATCH] mm: per-cgroup memory reclaim stats

2017-05-19 Thread Michal Hocko
<han...@cmpxchg.org> > Cc: Johannes Weiner <han...@cmpxchg.org> > Cc: Tejun Heo <t...@kernel.org> > Cc: Li Zefan <lize...@huawei.com> > Cc: Michal Hocko <mho...@kernel.org> > Cc: Vladimir Davydov <vdavydov@gmail.com> > Cc: cgro...@vger.kernel.

Re: [RFC PATCH] mm, oom: cgroup-aware OOM-killer

2017-05-18 Thread Michal Hocko
One possibility was to allow bpf like mechanisms. Could you explore that path? -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH] mm, oom: cgroup-aware OOM-killer

2017-05-23 Thread Michal Hocko
ree, that we can have some option to disable the cgroup-aware OOM at all, > mostly for backward-compatibility. But I don't think it should be a > per-cgroup configuration option, which we will support forever. I can clearly see a demand for "this is definitely more important container than

Re: [RFC PATCH] mm, oom: cgroup-aware OOM-killer

2017-05-25 Thread Michal Hocko
On Tue 23-05-17 09:25:44, Johannes Weiner wrote: > On Tue, May 23, 2017 at 09:07:47AM +0200, Michal Hocko wrote: > > On Mon 22-05-17 18:01:16, Roman Gushchin wrote: [...] > > > How to react on an OOM - is definitely a policy, which depends > > > on the workload. Nothin

Re: [RFC PATCH] mm, oom: cgroup-aware OOM-killer

2017-06-02 Thread Michal Hocko
On Wed 31-05-17 14:01:45, Johannes Weiner wrote: > On Wed, May 31, 2017 at 06:25:04PM +0200, Michal Hocko wrote: > > On Thu 25-05-17 13:08:05, Johannes Weiner wrote: > > > Everything the user would want to dynamically program in the kernel, > > > say with bpf,

Re: [RFC PATCH] mm, oom: cgroup-aware OOM-killer

2017-06-05 Thread Michal Hocko
On Fri 02-06-17 16:18:52, Roman Gushchin wrote: > On Fri, Jun 02, 2017 at 10:43:33AM +0200, Michal Hocko wrote: > > On Wed 31-05-17 14:01:45, Johannes Weiner wrote: > > > On Wed, May 31, 2017 at 06:25:04PM +0200, Michal Hocko wrote: > > > > > > + /* > &g

Re: [RFC PATCH] mm, oom: cgroup-aware OOM-killer

2017-05-19 Thread Michal Hocko
On Thu 18-05-17 14:11:17, Johannes Weiner wrote: > On Thu, May 18, 2017 at 07:30:04PM +0200, Michal Hocko wrote: > > On Thu 18-05-17 17:28:04, Roman Gushchin wrote: > > > Traditionally, the OOM killer is operating on a process level. > > > Under oom conditions, it finds

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-15 Thread Michal Hocko
On Thu 14-09-17 09:05:48, Roman Gushchin wrote: > On Thu, Sep 14, 2017 at 03:40:14PM +0200, Michal Hocko wrote: > > On Wed 13-09-17 14:56:07, Roman Gushchin wrote: > > > On Wed, Sep 13, 2017 at 02:29:14PM +0200, Michal Hocko wrote: > > [...] > > > > I stron

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-14 Thread Michal Hocko
On Wed 13-09-17 13:46:08, David Rientjes wrote: > On Wed, 13 Sep 2017, Michal Hocko wrote: > > > > > This patchset makes the OOM killer cgroup-aware. > > > > > > > > v8: > > > > - Do not kill tasks with OOM_SCORE_ADJ -1000 > >

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-14 Thread Michal Hocko
On Wed 13-09-17 14:56:07, Roman Gushchin wrote: > On Wed, Sep 13, 2017 at 02:29:14PM +0200, Michal Hocko wrote: [...] > > I strongly believe that comparing only leaf memcgs > > is more straightforward and it doesn't lead to unexpected results as > > mentioned before (kil

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-18 Thread Michal Hocko
On Fri 15-09-17 08:23:01, Roman Gushchin wrote: > On Fri, Sep 15, 2017 at 12:58:26PM +0200, Michal Hocko wrote: > > On Thu 14-09-17 09:05:48, Roman Gushchin wrote: > > > On Thu, Sep 14, 2017 at 03:40:14PM +0200, Michal Hocko wrote: > > > > On Wed 13-09-17

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-18 Thread Michal Hocko
; cgroups and they can own memory.oom_priority for their own subcontainers, > this becomes quite powerful so they can define their own oom priorities. > Otherwise, they can easily override the oom priorities of other cgroups. Could you be more speicific about your usecase? What would

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-18 Thread Michal Hocko
are on different levels > (or in different subtrees). Well, I have given you one that doesn't sounds completely insane to me in other email. You may need an intermediate level for other than memcg controller. The whole concept of significance of the hierarchy level seems really odd to me. Or am I wrong here? -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v8 1/4] mm, oom: refactor the oom_kill_process() function

2017-09-14 Thread Michal Hocko
im cgroup. We don't need to print > the debug information for the each task, as well as play > with task selection (considering task's children), > so we can't use the existing oom_kill_process(). > > Signed-off-by: Roman Gushchin <g...@fb.com> > Cc: Michal Hocko <mho...@kern

Re: [v8 0/4] cgroup-aware OOM killer

2017-10-02 Thread Michal Hocko
On Mon 02-10-17 13:24:25, Shakeel Butt wrote: > On Mon, Oct 2, 2017 at 12:56 PM, Michal Hocko <mho...@kernel.org> wrote: > > On Mon 02-10-17 12:45:18, Shakeel Butt wrote: > >> > I am sorry to cut the rest of your proposal because it simply goes over > >> > t

Re: [v8 0/4] cgroup-aware OOM killer

2017-10-02 Thread Michal Hocko
understand the proposal (from > reading thread, not patch) it does not. No it doesn't. It allows you to kill A (recursively) as the largest memory consumer. So, no, it cannot be used for prioritization, but again this is not yet the scope of the proposed solution. -- Michal Hocko SUSE Labs -- T

Re: [v9 3/5] mm, oom: cgroup-aware OOM killer

2017-10-04 Thread Michal Hocko
On Tue 03-10-17 07:35:59, Tejun Heo wrote: > Hello, Michal. > > On Tue, Oct 03, 2017 at 04:22:46PM +0200, Michal Hocko wrote: > > On Tue 03-10-17 15:08:41, Roman Gushchin wrote: > > > On Tue, Oct 03, 2017 at 03:36:23PM +0200, Michal Hocko wrote: > > [...] >

Re: [v11 3/6] mm, oom: cgroup-aware OOM killer

2017-10-10 Thread Michal Hocko
delay = true; > > + goto out; > > + } > > + > > select_bad_process(oc); > > This is racy because mem_cgroup_select_oom_victim() found an eligible > oc->chosen_memcg that is not INFLIGHT_VICTIM with at least one eligible > process but mem_

Re: [v10 5/6] mm, oom: add cgroup v2 mount option for cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
the interface part but I disagree with making it default just because v2 is not largerly adopted yet. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v10 5/6] mm, oom: add cgroup v2 mount option for cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
On Thu 05-10-17 14:41:13, Roman Gushchin wrote: > On Thu, Oct 05, 2017 at 03:14:19PM +0200, Michal Hocko wrote: > > On Wed 04-10-17 16:04:53, Johannes Weiner wrote: > > [...] > > > That will silently ignore what the user writes to the memory.oom_group > > > control

Re: [v10 4/6] mm, oom: introduce memory.oom_group

2017-10-05 Thread Michal Hocko
On Thu 05-10-17 13:32:14, Roman Gushchin wrote: > On Thu, Oct 05, 2017 at 02:06:49PM +0200, Michal Hocko wrote: > > On Wed 04-10-17 16:46:36, Roman Gushchin wrote: > > > The cgroup-aware OOM killer treats leaf memory cgroups as memory > > > consumption entities and pe

Re: [v10 3/6] mm, oom: cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
ext patch would probably make sense. Although, us reviewers have > been made aware of this now, so I don't feel strongly about it. Won't > make much of a difference once the patches are merged. I think it would be better to move it because it will be less confusing that way. Especially for those who are going to read git history in order to understand why this is needed. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v10 4/6] mm, oom: introduce memory.oom_group

2017-10-05 Thread Michal Hocko
uate_task, oc); > + > + if (oc->chosen_task == NULL || > + oc->chosen_task == INFLIGHT_VICTIM) > + goto out; How can this happen? There shouldn't be any INFLIGHT_VICTIM in our memcg because we have checked for that already. I can

Re: [v10 3/6] mm, oom: cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
all > > oom_evaluate_memcg() for offlined memcgs. > > Sounds like a good optimization, which can be done on top of the current > patchset. You could achive this by checking whether a memcg has tasks rather than explicitly checking for children memcgs as I've suggested already. --

Re: [v10 3/6] mm, oom: cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
anks for separating the group_oom part. This is getting in the mergeable state. I will ack it once the suggested fixes are folded in. There is some clean up potential on top (I find the oc->chosen_memcg quite ugly and will post a patch on top of yours) but that can be done later. > Signed-of

Re: [v10 5/6] mm, oom: add cgroup v2 mount option for cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
On Thu 05-10-17 10:54:01, Johannes Weiner wrote: > On Thu, Oct 05, 2017 at 03:14:19PM +0200, Michal Hocko wrote: > > On Wed 04-10-17 16:04:53, Johannes Weiner wrote: > > [...] > > > That will silently ignore what the user writes to the memory.oom_group > > >

Re: [v11 4/6] mm, oom: introduce memory.oom_group

2017-10-05 Thread Michal Hocko
oc->chosen_memcg = group; + if (score > oc->chosen_points) { + oc->chosen_points = score; + oc->chosen_memcg = iter; } } -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v11 3/6] mm, oom: cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
. > > The root cgroup is treated as a leaf memory cgroup, so it's score > is compared with leaf memory cgroups. > Due to memcg statistics implementation a special algorithm > is used for estimating it's oom_score: we define it as maximum > oom_score of the belonging tasks. &g

Re: [v11 4/6] mm, oom: introduce memory.oom_group

2017-10-05 Thread Michal Hocko
er defined configuration might lead to data corruptions or other > misbehavior. > > The default value is 0. I still believe that oc->chosen_task == INFLIGHT_VICTIM check in oom_kill_memcg_victim should go away. > > Signed-off-by: Roman Gushchin <g...@fb.com> > Cc: Mi

Re: [v10 3/6] mm, oom: cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
knob on top of that if you need. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v11 4/6] mm, oom: introduce memory.oom_group

2017-10-06 Thread Michal Hocko
On Fri 06-10-17 13:04:35, Roman Gushchin wrote: > On Thu, Oct 05, 2017 at 04:31:04PM +0200, Michal Hocko wrote: > > Btw. here is how I would do the recursive oom badness. The diff is not > > the nicest one because there is some code moving but the resulting code > > is sm

Re: [PATCH 2/2] mm: rename page dtor functions to {compound,huge,transhuge}_page__dtor

2017-10-17 Thread Michal Hocko
On Tue 17-10-17 14:22:14, Kirill A. Shutemov wrote: > On Tue, Oct 17, 2017 at 12:22:03PM +0200, Michal Hocko wrote: > > On Mon 16-10-17 17:19:17, changbin...@intel.com wrote: > > > From: Changbin Du <changbin...@intel.com> > > > > > > The current

Re: [PATCH 1/2] mm, thp: introduce dedicated transparent huge page allocation interfaces

2017-10-17 Thread Michal Hocko
On Tue 17-10-17 12:20:52, Michal Hocko wrote: > [CC Kirill] now for real > On Mon 16-10-17 17:19:16, changbin...@intel.com wrote: > > From: Changbin Du <changbin...@intel.com> > > > > This patch introduced 4 new interfaces to allocate a prep

Re: [PATCH 2/2] mm: rename page dtor functions to {compound,huge,transhuge}_page__dtor

2017-10-17 Thread Michal Hocko
* the reservation was consumed when the page was allocated. >* We clear the PagePrivate flag now so that the global > - * reserve count will not be incremented in free_huge_page. > + * reserve count will not be incremented in huge_page_dtor. >* The reservation map will still indicate the reservation >* was consumed and possibly prevent later page allocation. >* This is better than leaking a global reservation. If no > -- > 2.7.4 > > -- > To unsubscribe, send a message with 'unsubscribe linux-mm' in > the body to majord...@kvack.org. For more info on Linux MM, > see: http://www.linux-mm.org/ . > Don't email: mailto:"d...@kvack.org;> em...@kvack.org -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] mm, thp: introduce dedicated transparent huge page allocation interfaces

2017-10-17 Thread Michal Hocko
; - HPAGE_PMD_ORDER); > + new_page = alloc_transhuge_page_node(node, > + (GFP_TRANSHUGE_LIGHT | __GFP_THISNODE)); > if (!new_page) > goto out_fail; > - prep_transhuge_page(new_page); > > isolated = numamigrate_isolate_page(pgd

Re: [v11 3/6] mm, oom: cgroup-aware OOM killer

2017-10-11 Thread Michal Hocko
s been stated several times already that future improvements are possible and cover what you have described already. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info

Re: [v7 2/5] mm, oom: cgroup-aware OOM killer

2017-09-11 Thread Michal Hocko
dness does for the regular oom killer. Or do you have anything else in mind? -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v7 5/5] mm, oom: cgroup v2 mount option to disable cgroup-aware OOM killer

2017-09-11 Thread Michal Hocko
On Thu 07-09-17 12:14:57, Johannes Weiner wrote: > On Wed, Sep 06, 2017 at 10:28:59AM +0200, Michal Hocko wrote: > > On Tue 05-09-17 17:53:44, Johannes Weiner wrote: > > > The cgroup-awareness in the OOM killer is exactly the same thing. It > > > should have been the

Re: [v7 5/5] mm, oom: cgroup v2 mount option to disable cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
On Tue 05-09-17 17:53:44, Johannes Weiner wrote: > On Tue, Sep 05, 2017 at 03:44:12PM +0200, Michal Hocko wrote: > > Why is this an opt out rather than opt-in? IMHO the original oom logic > > should be preserved by default and specific workloads should opt in for > > t

Re: [v7 5/5] mm, oom: cgroup v2 mount option to disable cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
On Tue 05-09-17 20:16:09, Roman Gushchin wrote: > On Tue, Sep 05, 2017 at 05:12:51PM +0200, Michal Hocko wrote: [...] > > > Then we should probably hide corresponding > > > cgroup interface (oom_group and oom_priority knobs) by default, > > > and it feels as unnecessa

Re: [v7 2/5] mm, oom: cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
On Tue 05-09-17 21:23:57, Roman Gushchin wrote: > On Tue, Sep 05, 2017 at 04:57:00PM +0200, Michal Hocko wrote: [...] > > > @@ -810,6 +810,9 @@ static void __oom_kill_process(struct task_struct > > > *victim) > > > struct mm_struct *mm; &g

Re: [v7 2/5] mm, oom: cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
On Tue 05-09-17 21:23:57, Roman Gushchin wrote: > On Tue, Sep 05, 2017 at 04:57:00PM +0200, Michal Hocko wrote: [...] > > Hmm. The changelog says "By default, it will look for the biggest leaf > > cgroup, and kill the largest task inside." But you are accumulating >

Re: [v7 5/5] mm, oom: cgroup v2 mount option to disable cgroup-aware OOM killer

2017-09-05 Thread Michal Hocko
On Tue 05-09-17 15:30:21, Roman Gushchin wrote: > On Tue, Sep 05, 2017 at 03:44:12PM +0200, Michal Hocko wrote: [...] > > Why is this an opt out rather than opt-in? IMHO the original oom logic > > should be preserved by default and specific workloads should opt in for > > t

Re: [v7 2/5] mm, oom: cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
On Wed 06-09-17 13:57:50, Roman Gushchin wrote: > On Wed, Sep 06, 2017 at 10:31:58AM +0200, Michal Hocko wrote: > > On Tue 05-09-17 21:23:57, Roman Gushchin wrote: > > > On Tue, Sep 05, 2017 at 04:57:00PM +0200, Michal Hocko wrote: > > [...] > > > >

Re: [v7 2/5] mm, oom: cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
e are special cases of course but we should target general use first. Kill the whole memcg is a really useful feature on its own for proper container cleanup. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to m

Re: [v7 5/5] mm, oom: cgroup v2 mount option to disable cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
uses it. > We want to completely deprecate it at some point. > > To make a first step towards deprecation, let's warn potential > users about deprecation plans. > > Signed-off-by: Roman Gushchin <g...@fb.com> > Cc: Andrew Morton <a...@linux-foundation.org> > Cc:

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-13 Thread Michal Hocko
eaf memcgs is more straightforward and it doesn't lead to unexpected results as mentioned before (kill a small memcg which is a part of the larger sub-hierarchy). I didn't get to read the new version of this series yet and hope to get to it soon. -- Michal Hocko SUSE Labs -- To unsubscribe from this l

Re: [v8 3/4] mm, oom: add cgroup v2 mount option for cgroup-aware OOM killer

2017-09-13 Thread Michal Hocko
tion is should it be opt-in or opt-out option. > Personally, I would prefer opt-out, but Michal has a very strong opinion here. Yes I still strongly believe this has to be opt-in. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-26 Thread Michal Hocko
On Tue 26-09-17 13:13:00, Roman Gushchin wrote: > On Tue, Sep 26, 2017 at 01:21:34PM +0200, Michal Hocko wrote: > > On Tue 26-09-17 11:59:25, Roman Gushchin wrote: > > > On Mon, Sep 25, 2017 at 10:25:21PM +0200, Michal Hocko wrote: > > > > On Mon 25-09-17

Re: [PATCH] mm,hugetlb,migration: don't migrate kernelcore hugepages

2017-10-02 Thread Michal Hocko
On Mon 02-10-17 16:06:33, Alexandru Moise wrote: > On Mon, Oct 02, 2017 at 02:54:32PM +0200, Michal Hocko wrote: > > On Mon 02-10-17 00:51:11, Alexandru Moise wrote: > > > This attempts to bring more flexibility to how hugepages are allocated > > > by making it possibl

Re: [v8 0/4] cgroup-aware OOM killer

2017-10-02 Thread Michal Hocko
On Mon 02-10-17 13:47:12, Roman Gushchin wrote: > On Mon, Oct 02, 2017 at 02:24:34PM +0200, Michal Hocko wrote: [...] > > I believe the latest version (v9) looks sensible from the semantic point > > of view and we should focus on making it into a mergeable shape. > > The onl

Re: [PATCH] mm,hugetlb,migration: don't migrate kernelcore hugepages

2017-10-02 Thread Michal Hocko
s are not really migratable which should be the only criterion. Hugetlb pages are no different from other migratable pages in that regards. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v8 0/4] cgroup-aware OOM killer

2017-10-02 Thread Michal Hocko
module based selection. But let's start simple with the most basic scenario first with a most sensible semantic implemented. I believe the latest version (v9) looks sensible from the semantic point of view and we should focus on making it into a mergeable shape. -- Michal Hocko SUSE Labs -- To u

Re: [PATCH] mm,hugetlb,migration: don't migrate kernelcore hugepages

2017-10-02 Thread Michal Hocko
On Mon 02-10-17 17:06:38, Alexandru Moise wrote: > On Mon, Oct 02, 2017 at 04:27:17PM +0200, Michal Hocko wrote: > > On Mon 02-10-17 16:06:33, Alexandru Moise wrote: > > > On Mon, Oct 02, 2017 at 02:54:32PM +0200, Michal Hocko wrote: > > > > On Mon 02-10-17 0

Re: [v8 0/4] cgroup-aware OOM killer

2017-10-02 Thread Michal Hocko
would really appreciate to focus on making the step 1 done before diverging into details about potential improvements and a better control over the selection. This whole thing is an opt-in so there is a no risk of a regression. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: se

Re: [v9 4/5] mm, oom: add cgroup v2 mount option for cgroup-aware OOM killer

2017-10-03 Thread Michal Hocko
line argument would fit better IMHO. > Signed-off-by: Roman Gushchin <g...@fb.com> > Cc: Michal Hocko <mho...@kernel.org> > Cc: Vladimir Davydov <vdavydov@gmail.com> > Cc: Johannes Weiner <han...@cmpxchg.org> > Cc: Tetsuo Handa <penguin-ker...@i-love

Re: [v9 2/5] mm: implement mem_cgroup_scan_tasks() for the root memory cgroup

2017-10-03 Thread Michal Hocko
is just a preparatory work for later changes. > Signed-off-by: Roman Gushchin <g...@fb.com> > Cc: Michal Hocko <mho...@kernel.org> > Cc: Vladimir Davydov <vdavydov@gmail.com> > Cc: Johannes Weiner <han...@cmpxchg.org> > Cc: Tetsuo Handa <penguin-ker...

Re: [v9 3/5] mm, oom: cgroup-aware OOM killer

2017-10-03 Thread Michal Hocko
footprint */ > + oc->chosen_points = 0; > + oc->chosen_task = NULL; > + mem_cgroup_scan_tasks(oc->chosen_memcg, oom_evaluate_task, oc); > + > + if (oc->chosen_task == NULL || oc->chosen_task == INFLIGHT_VICTIM) > + goto out; > + >

Re: [PATCH] mm,hugetlb,migration: don't migrate kernelcore hugepages

2017-10-03 Thread Michal Hocko
On Tue 03-10-17 07:42:25, Alexandru Moise wrote: > On Mon, Oct 02, 2017 at 06:15:00PM +0200, Michal Hocko wrote: [...] > > I really fail to see why kernel vs. movable zones play any role here. > > Zones should be mostly an implementation detail which userspace > > should

Re: [v9 3/5] mm, oom: cgroup-aware OOM killer

2017-10-03 Thread Michal Hocko
On Tue 03-10-17 13:37:21, Roman Gushchin wrote: > On Tue, Oct 03, 2017 at 01:48:48PM +0200, Michal Hocko wrote: [...] > > Wrt. to the implicit inheritance you brought up in a separate email > > thread [1]. Let me quote > > : after some additional thinking I don't think

  1   2   >