Re: [PATCH] mremap: Avoid TLB flushing anonymous pages that are not in swap cache

2018-06-06 Thread Michal Hocko
add some perf numbers for the specific workload which triggered this (a mremap heavy workload which is real unfortunately). > Signed-off-by: Mel Gorman Acked-by: Michal Hocko > --- > mm/mremap.c | 45 - > 1 file changed, 40 insertions(+), 5 delet

Re: [PATCH v13 0/7] cgroup-aware OOM killer

2018-06-05 Thread Michal Hocko
On Tue 05-06-18 13:47:29, Michal Hocko wrote: > It seems that this is still in limbo mostly because of David's concerns. > So let me reiterate them and provide my POV once more (and the last > time) just to help Andrew make a decision: Sorry, I forgot to add reference to the email with

Re: [PATCH 2/2] mm: don't skip memory guarantee calculations

2018-06-05 Thread Michal Hocko
On Tue 05-06-18 11:15:45, Roman Gushchin wrote: > On Tue, Jun 05, 2018 at 11:03:49AM +0200, Michal Hocko wrote: > > On Mon 04-06-18 17:23:06, Roman Gushchin wrote: > > [...] > > > I'm happy to discuss any concrete issues/concerns, but I really see > > > no reason

Re: [PATCH 2/2] mm: don't skip memory guarantee calculations

2018-06-05 Thread Michal Hocko
ly prefer to see the whole thing in one series to have a better picture. -- Michal Hocko SUSE Labs

Re: [rfc patch] mm, oom: fix unnecessary killing of additional processes

2018-06-05 Thread Michal Hocko
On Mon 04-06-18 21:25:39, David Rientjes wrote: > On Fri, 1 Jun 2018, Michal Hocko wrote: > > > > We've discussed the mm > > > having a single blockable mmu notifier. Regardless of how we arrive at > > > the point where the oom reaper can't free memory, which c

Re: [PATCH 0/4] exit: Make unlikely case in mm_update_next_owner() more scalable

2018-06-05 Thread Michal Hocko
of course. -- Michal Hocko SUSE Labs

Re: [PATCH V4] mlx4_core: allocate ICM memory in page size chunks

2018-06-04 Thread Michal Hocko
On Thu 31-05-18 11:10:22, Michal Hocko wrote: > On Thu 31-05-18 10:55:32, Michal Hocko wrote: > > On Thu 31-05-18 04:35:31, Eric Dumazet wrote: > [...] > > > I merely copied/pasted from alloc_skb_with_frags() :/ > > > > I will have a look at it. Thank

Re: [RFC][PATCH 1/2] memcg: Ensure every task that uses an mm is in the same memory cgroup

2018-06-04 Thread Michal Hocko
ding concept and depend on memory migration which doesn't really work? -- Michal Hocko SUSE Labs

Re: [PATCH 2/2] mm: don't skip memory guarantee calculations

2018-06-04 Thread Michal Hocko
whole thing after the merge window, please? As I've said earlier I am not even sure we really want to have a hard guarantee once we decided to go with low limit. So a very good reasoning should be added for the whole thing. Thanks! -- Michal Hocko SUSE Labs

Re: [PATCH 1/2] mm: propagate memory effective protection on setting memory.min/low

2018-06-04 Thread Michal Hocko
se be explicit about the exact problem. Ideally with a memcg tree example. > Signed-off-by: Roman Gushchin > Cc: Johannes Weiner > Cc: Michal Hocko > Cc: Vladimir Davydov > Cc: Greg Thelen > Cc: Tejun Heo > Cc: Andrew Morton > --- > mm/memcontrol.c | 14 ++

Re: [PATCH v7 2/2] Refactor part of the oom report in dump_header

2018-06-04 Thread Michal Hocko
, I will use the pr_cont_cgroup_name() to print origin and kill > memcg's name. I hope David will not have other opinions :) As I've said this can be always added on top pressuming there is a good justification. -- Michal Hocko SUSE Labs

Re: [PATCH 2/5] lib/rhashtable: guarantee initial hashtable allocation

2018-06-04 Thread Michal Hocko
t; tbl = kvzalloc(size, gfp); Just a heads up. This is not longer needed after http://lkml.kernel.org/r/20180601115329.27807-1-mho...@kernel.org is applied. Moreover it will conflict at that spot. -- Michal Hocko SUSE Labs

Re: [PATCH] mm: Add conditions to avoid out-of-bounds

2018-06-04 Thread Michal Hocko
On Mon 04-06-18 18:37:35, nixiaoming wrote: > In the function memcg_init_list_lru > if call goto fail when i == 0, will cause out-of-bounds at lru->node[i] How? All I can see is that the fail path does for (i = i - 1; i >= 0; i--) so it will not do anything for i=0. -- Micha

Re: [PATCH v7 2/2] Refactor part of the oom report in dump_header

2018-06-04 Thread Michal Hocko
into a single print because I am not sure the benefit is really worth it. Maybe others will though. -- Michal Hocko SUSE Labs

Re: [PATCH 0/4] exit: Make unlikely case in mm_update_next_owner() more scalable

2018-06-04 Thread Michal Hocko
On Fri 01-06-18 10:25:59, Eric W. Biederman wrote: > Michal Hocko writes: > > > On Fri 01-06-18 09:32:42, Eric W. Biederman wrote: > >> Michal Hocko writes: > > [...] > >> > Group leader exiting early without tearing down the whole thread > >> &

Re: [PATCH v7 2/2] Refactor part of the oom report in dump_header

2018-06-04 Thread Michal Hocko
I follow the advices of David Rientjes and Michal Hocko, and refactor > part of the oom report in a backwards compatible way. After this patch, > users can get the memcg's path from the oom report and check the certain > container more quickly. I have earlier suggested that you split t

Re: [PATCH v7 1/2] Add an array of const char and enum oom_constraint in memcontrol.h

2018-06-04 Thread Michal Hocko
k_struct *p, struct > mem_cgroup *memcg, > return points > 0 ? points : 1; > } > > -enum oom_constraint { > - CONSTRAINT_NONE, > - CONSTRAINT_CPUSET, > - CONSTRAINT_MEMORY_POLICY, > - CONSTRAINT_MEMCG, > -}; > - > /* > * Determine the type of allocation constraint. > */ > -- > 2.14.1 -- Michal Hocko SUSE Labs

Re: [PATCH] mm: clean up page_is_poisoned in linux/mm.h

2018-06-04 Thread Michal Hocko
On Sun 03-06-18 12:22:18, kpark3...@gmail.com wrote: > From: Sahara > > When bd33ef36("mm: enable page poisoning early at boot") got rid of > the PAGE_EXT_DEBUG_POISON, page_is_poisoned in the header left > behind. This patch cleans up the leftovers under the table. Ack

Re: [PATCH] mm: kvmalloc does not fallback to vmalloc for incompatible gfp flags

2018-06-04 Thread Michal Hocko
On Sat 02-06-18 09:43:56, Linus Torvalds wrote: > On Fri, Jun 1, 2018 at 4:53 AM Michal Hocko wrote: > > > > for more context. Linus has pointed out [1] that our (well mine) > > insisting on GFP_KERNEL compatible gfp flags for kvmalloc* can actually > > lead to

Re: [PATCH 0/4] exit: Make unlikely case in mm_update_next_owner() more scalable

2018-06-01 Thread Michal Hocko
On Fri 01-06-18 09:32:42, Eric W. Biederman wrote: > Michal Hocko writes: [...] > > Group leader exiting early without tearing down the whole thread > > group should be quite rare as well. No question that somebody might do > > that on purpose though... > > The

Re: [PATCH 0/2] mm->owner to mm->memcg fixes

2018-06-01 Thread Michal Hocko
On Thu 31-05-18 13:41:38, Eric W. Biederman wrote: > Michal Hocko writes: > > > On Thu 24-05-18 14:16:35, Andrew Morton wrote: > >> On Thu, 24 May 2018 13:10:02 +0200 Michal Hocko wrote: > >> > >> > I would really prefer and app

Re: [PATCH 0/4] exit: Make unlikely case in mm_update_next_owner() more scalable

2018-06-01 Thread Michal Hocko
On Thu 31-05-18 20:07:28, Eric W. Biederman wrote: > Michal Hocko writes: > > > On Thu 26-04-18 14:00:19, Kirill Tkhai wrote: > >> This function searches for a new mm owner in children and siblings, > >> and then iterates over all processes in the system in unlike

Re: mlock() confusing 1 half of system RAM limit

2018-06-01 Thread Michal Hocko
t; EFAULT -> ENOMEM on the way. -- Michal Hocko SUSE Labs

Re: mlock() confusing 1 half of system RAM limit

2018-06-01 Thread Michal Hocko
_forced = 0 > kernel.shmall = 18446744073692774399 > kernel.shmmax = 18446744073692774399 > kernel.shmmni = 4096 > > Any ideas? shm_open uses tmpfs/shmem under the cover and that has the internal limit as explained above. -- Michal Hocko SUSE Labs

[PATCH] mm: kvmalloc does not fallback to vmalloc for incompatible gfp flags

2018-06-01 Thread Michal Hocko
From: Michal Hocko kvmalloc warned about incompatible gfp_mask to catch abusers (mostly GFP_NOFS) with an intention that this will motivate authors of the code to fix those. Linus argues that this just motivates people to do even more hacks like if (gfp == GFP_KERNEL

Re: mlock() confusing 1 half of system RAM limit

2018-06-01 Thread Michal Hocko
resp. on mmap of shmem file. You are likely to hit the shmem limit. This is half of the available memory by default. -- Michal Hocko SUSE Labs

Re: deadlock during writeback when using f2fs filesystem

2018-06-01 Thread Michal Hocko
S | __GFP_ZERO) > #define GFP_F2FS_HIGH_ZERO (GFP_NOFS | __GFP_ZERO | __GFP_HIGHMEM) > +#define GFP_F2FS_NODE_MAPPING (GFP_NOWAIT | __GFP_IO | __GFP_ZERO) > > Thanks, > Sahitya. > -- > -- > Sent by a consultant of the Qualcomm Innovation Center, Inc. > The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum. -- Michal Hocko SUSE Labs

Re: [rfc patch] mm, oom: fix unnecessary killing of additional processes

2018-06-01 Thread Michal Hocko
On Thu 31-05-18 14:16:34, David Rientjes wrote: > On Thu, 31 May 2018, Michal Hocko wrote: > > > > It's not a random timeout, it's sufficiently long such that we don't oom > > > kill several processes needlessly in the very rare case where oom > > > livel

Re: [PATCH 3/6] lib/bucket_locks: use kvmalloc_array()

2018-05-31 Thread Michal Hocko
On Thu 31-05-18 10:01:51, Linus Torvalds wrote: > On Wed, May 30, 2018 at 2:42 AM Michal Hocko wrote: > > > > That being sad, if you believe that silently fixing up a code like that > > is a good idea we can do the following of course: > > Ack. > > Except for: &

Re: [PATCH 2/2] fs, elf: drop MAP_FIXED usage from elf_map

2018-05-31 Thread Michal Hocko
. How come the original content is not needed anymore? Maybe the first section should be 0x1000 rather than 0x169c? I am not an expert on the load linkers myself so I cannot really answer this question. Please note that ppc had something similar. See ad55eac74f20 ("elf: enforce MAP_FIXED on overlaying elf segments"). Maybe we need to sprinkle more of those at other places? -- Michal Hocko SUSE Labs

Re: [PATCH] memcg: force charge kmem counter too

2018-05-31 Thread Michal Hocko
On Thu 31-05-18 17:23:17, Minchan Kim wrote: > On Thu, May 31, 2018 at 08:56:42AM +0200, Michal Hocko wrote: > > On Thu 31-05-18 15:01:33, Minchan Kim wrote: > > > On Wed, May 30, 2018 at 11:14:33AM -0700, Shakeel Butt wrote: > > > > On Tue, May 29, 2018 a

Re: [PATCH] memcg: force charge kmem counter too

2018-05-31 Thread Michal Hocko
On Thu 31-05-18 15:01:33, Minchan Kim wrote: > On Wed, May 30, 2018 at 11:14:33AM -0700, Shakeel Butt wrote: > > On Tue, May 29, 2018 at 1:31 AM, Michal Hocko wrote: > > > On Mon 28-05-18 10:23:07, Shakeel Butt wrote: > > >> On Mon, May 28, 2018 at 2:11 AM, Mich

Re: [PATCH v6] Refactor part of the oom report in dump_header

2018-05-31 Thread Michal Hocko
On Sun 27-05-18 10:32:31, ufo19890607 wrote: > The dump_header does not print the memcg's name when the system > oom happened, so users cannot locate the certain container which > contains the task that has been killed by the oom killer. > > I follow the advices of David Rientjes a

Re: [PATCH v4] Print the memcg's name when system-wide OOM happened

2018-05-31 Thread Michal Hocko
ion [1] seems to be doing the right thing in that regards AFAICS. It has some other issues though. Can we drop the current code from the mmotm tree and start over? [1] http://lkml.kernel.org/r/1527413551-5982-1-git-send-email-ufo19890...@gmail.com -- Michal Hocko SUSE Labs

Re: [rfc patch] mm, oom: fix unnecessary killing of additional processes

2018-05-31 Thread Michal Hocko
On Wed 30-05-18 14:06:51, David Rientjes wrote: > On Mon, 28 May 2018, Michal Hocko wrote: > > > > That's not sufficient since the oom reaper is also not able to oom reap > > > if > > > the mm has blockable mmu notifiers or all memory is shared filebacked >

Re: [PATCH 2/2] fs, elf: drop MAP_FIXED usage from elf_map

2018-05-30 Thread Michal Hocko
On Wed 30-05-18 08:00:29, Mike Kravetz wrote: > On 05/30/2018 01:02 AM, Michal Hocko wrote: > > On Tue 29-05-18 15:21:14, Mike Kravetz wrote: > >> Just a quick heads up. I noticed a change in libhugetlbfs testing starting > >> with v4.17-rc1. > >>

Re: [PATCH 0/2] mm->owner to mm->memcg fixes

2018-05-30 Thread Michal Hocko
On Thu 24-05-18 14:16:35, Andrew Morton wrote: > On Thu, 24 May 2018 13:10:02 +0200 Michal Hocko wrote: > > > I would really prefer and appreciate a repost with all the fixes folded > > in. > > [1/2] Thanks Andrew and sorry it took so long! This seems to be missing th

Re: [PATCH 0/2] mm->owner to mm->memcg fixes

2018-05-30 Thread Michal Hocko
treat the mm as belong to the root memory cgroup. Does that mean that a malicious user can construct such a task and runaway from its limits? -- Michal Hocko SUSE Labs

Re: [PATCH 2/2] fs, elf: drop MAP_FIXED usage from elf_map

2018-05-30 Thread Michal Hocko
and, will do some analysis to see exactly what is happening. I am definitely interested about further details. Are there any messages in the kernel log? -- Michal Hocko SUSE Labs

Re: [PATCH 3/6] lib/bucket_locks: use kvmalloc_array()

2018-05-30 Thread Michal Hocko
an do the following of course: >From c1a098e809a109800f9cfa63cb27fe9a78f3f316 Mon Sep 17 00:00:00 2001 From: Michal Hocko Date: Wed, 30 May 2018 09:34:39 +0200 Subject: [PATCH] mm: kvmalloc does not fallback to vmalloc for incompatible gfp flags kvmalloc warned about incompatible gfp_mask to catch abusers (mostly GFP_NOFS) w

Re: [PATCH 3/6] lib/bucket_locks: use kvmalloc_array()

2018-05-29 Thread Michal Hocko
On Tue 29-05-18 16:43:17, Michal Hocko wrote: > On Thu 24-05-18 14:37:36, Linus Torvalds wrote: > > On Thu, May 24, 2018 at 2:28 PM Davidlohr Bueso wrote: > > > > > if (gfpflags_allow_blocking(gfp)) > > > - tlocks = kvmall

Re: [PATCH 3/6] lib/bucket_locks: use kvmalloc_array()

2018-05-29 Thread Michal Hocko
too much thinking or use the scope NOFS/NOIO API)? A warn_on tends to be rather harsh but effective way to push maintainers fix their broken code... -- Michal Hocko SUSE Labs

Re: [PATCH v2] doc: document scope NOFS, NOIO APIs

2018-05-29 Thread Michal Hocko
On Tue 29-05-18 05:51:58, Jonathan Corbet wrote: > On Tue, 29 May 2018 10:26:44 +0200 > Michal Hocko wrote: > > > Although the api is documented in the source code Ted has pointed out > > that there is no mention in the core-api Documentation and there are > > p

Re: [LKP] [lkp-robot] [mm] e27be240df: will-it-scale.per_process_ops -27.2% regression

2018-05-29 Thread Michal Hocko
t. In the meantime, it might not be > a good idea to ensure that since stat_cpu should be an always_read field > while moving_account will be modified when needed. > > Or any idea what might be the cause? Thanks. Can you actually prepare a patch with all these numbers and a big fat comment in the structure to keep the most hot counters close to moving_account. Maybe we want to re-organize this some more and pull move_lock* out of the sensitive cache line because they are a slow path stuff. We would have more stasts in the same cache line then. What do you think? -- Michal Hocko SUSE Labs

Re: [PATCH] memcg: force charge kmem counter too

2018-05-29 Thread Michal Hocko
On Mon 28-05-18 10:23:07, Shakeel Butt wrote: > On Mon, May 28, 2018 at 2:11 AM, Michal Hocko wrote: > Though is there a precedence where the broken feature is not fixed > because an alternative is available? Well, I can see how breaking GFP_NOFAIL semantic is problematic, on the othe

Re: [LKP] [lkp-robot] [mm, memcontrol] 309fe96bfc: vm-scalability.throughput +23.0% improvement

2018-05-29 Thread Michal Hocko
On Tue 29-05-18 16:11:27, Aaron Lu wrote: > On Tue, May 29, 2018 at 09:58:00AM +0200, Michal Hocko wrote: > > On Tue 29-05-18 03:15:51, Lu, Aaron wrote: > > > On Mon, 2018-05-28 at 14:03 +0200, Michal Hocko wrote: > > > > On Mon 28-05-18 19:

[PATCH v2] doc: document scope NOFS, NOIO APIs

2018-05-29 Thread Michal Hocko
From: Michal Hocko Although the api is documented in the source code Ted has pointed out that there is no mention in the core-api Documentation and there are people looking there to find answers how to use a specific API. Changes since v1 - add kerneldoc for the api - suggested by Johnatan

Re: [PATCH] doc: document scope NOFS, NOIO APIs

2018-05-29 Thread Michal Hocko
On Mon 28-05-18 10:21:00, Nikolay Borisov wrote: > > > On 25.05.2018 10:52, Michal Hocko wrote: > > On Thu 24-05-18 09:37:18, Randy Dunlap wrote: > >> On 05/24/2018 04:43 AM, Michal Hocko wrote: > > [...] > >>> +The traditional way to avoid this deadloc

Re: [PATCH] doc: document scope NOFS, NOIO APIs

2018-05-29 Thread Michal Hocko
On Mon 28-05-18 09:10:43, Randy Dunlap wrote: > On 05/28/2018 02:21 AM, Michal Hocko wrote: [...] > > +reclaim context or when a transaction context nesting would be possible > > +via reclaim. The corresponding restore function when the critical > > "The corr

Re: [PATCH] doc: document scope NOFS, NOIO APIs

2018-05-29 Thread Michal Hocko
On Tue 29-05-18 08:32:05, Dave Chinner wrote: > On Mon, May 28, 2018 at 11:19:23AM +0200, Michal Hocko wrote: > > On Mon 28-05-18 09:48:54, Dave Chinner wrote: > > > On Fri, May 25, 2018 at 10:16:24AM +0200, Michal Hocko wrote: > > > > On Fri 25-05-18 08:17:15, Dave C

Re: [LKP] [lkp-robot] [mm, memcontrol] 309fe96bfc: vm-scalability.throughput +23.0% improvement

2018-05-29 Thread Michal Hocko
On Tue 29-05-18 03:15:51, Lu, Aaron wrote: > On Mon, 2018-05-28 at 14:03 +0200, Michal Hocko wrote: > > On Mon 28-05-18 19:40:19, kernel test robot wrote: > > > > > > Greeting, > > > > > > FYI, we noticed a +23.0% improvement of v

Re: [External] Re: [RFC PATCH v2 00/12] get rid of GFP_ZONE_TABLE/BAD

2018-05-28 Thread Michal Hocko
On Fri 25-05-18 09:43:09, Huaisheng HS1 Ye wrote: > From: Michal Hocko [mailto:mho...@kernel.org] > Sent: Thursday, May 24, 2018 8:19 PM> > > > Let me try to reply your questions. > > > Exactly, GFP_ZONE_TABLE is too complicated. I think there are two > > &

Re: [rfc patch] mm, oom: fix unnecessary killing of additional processes

2018-05-28 Thread Michal Hocko
On Fri 25-05-18 12:36:08, David Rientjes wrote: > On Fri, 25 May 2018, Michal Hocko wrote: > > > > The oom reaper ensures forward progress by setting MMF_OOM_SKIP itself if > > > it cannot reap an mm. This can happen for a variety of reasons, > > > includi

Re: [PATCH] doc: document scope NOFS, NOIO APIs

2018-05-28 Thread Michal Hocko
On Mon 28-05-18 09:48:54, Dave Chinner wrote: > On Fri, May 25, 2018 at 10:16:24AM +0200, Michal Hocko wrote: > > On Fri 25-05-18 08:17:15, Dave Chinner wrote: > > > On Thu, May 24, 2018 at 01:43:41PM +0200, Michal Hocko wrote: > > [...] > > > > +FS/IO code th

Re: [patch] mm, hugetlb_cgroup: suppress SIGBUS when hugetlb_cgroup charge fails

2018-05-28 Thread Michal Hocko
ast gets SIGBUS. Hmm, so you expect that the killed task would simply return pages to the pool? Wouldn't that require to have a hugetlb cgroup OOM killer that would only care about hugetlb reservations of tasks? Is that worth all the effort and the additional code? -- Michal Hocko SUSE Labs

Re: [PATCH] memcg: force charge kmem counter too

2018-05-28 Thread Michal Hocko
e listing a directory. Bypassing the limit for NOFAIL > > allocations isn't going to fix those problem. > > I understand that fixing NOFAIL will not fix all other issues but it > still is better than current situation. IMHO we should keep fixing > kmem bit by bit. > > One crazy

Re: [RFC PATCH v2 00/12] get rid of GFP_ZONE_TABLE/BAD

2018-05-28 Thread Michal Hocko
On Fri 25-05-18 05:00:44, Matthew Wilcox wrote: > On Thu, May 24, 2018 at 05:29:43PM +0200, Michal Hocko wrote: > > > ie if we had more, > > > could we solve our pain by making them more generic? > > > > Well, if you have more you will consume more bi

Re: [PATCH] doc: document scope NOFS, NOIO APIs

2018-05-28 Thread Michal Hocko
On Sun 27-05-18 15:47:22, Mike Rapoport wrote: > On Fri, May 25, 2018 at 10:16:24AM +0200, Michal Hocko wrote: > > On Fri 25-05-18 08:17:15, Dave Chinner wrote: > > > On Thu, May 24, 2018 at 01:43:41PM +0200, Michal Hocko wrote: > > [...] > > > > +FS/IO code th

Re: [lkp-robot] [mm, memcontrol] 309fe96bfc: vm-scalability.throughput +23.0% improvement

2018-05-28 Thread Michal Hocko
m seeing something like that I am afraid. -- Michal Hocko SUSE Labs

Re: [PATCH] mm, page_alloc: do not break __GFP_THISNODE by zonelist reset

2018-05-28 Thread Michal Hocko
_node_id for __GFP_THISNODE is a clear bug because our task could have been migrated to a cpu on a different than requested node. Acked-by: Michal Hocko <mho...@suse.com> -- Michal Hocko SUSE Labs

Re: [patch] mm, hugetlb_cgroup: suppress SIGBUS when hugetlb_cgroup charge fails

2018-05-28 Thread Michal Hocko
_put: > if (map_chg || avoid_reserve) > hugepage_subpool_put_pages(spool, 1); > +out_reservation: > vma_end_reservation(h, vma, addr); > - return ERR_PTR(-ENOSPC); > +out: > + return ERR_PTR(ret); > } > > int alloc_bootmem_huge_page(struct hstate *h) -- Michal Hocko SUSE Labs

Re: [PATCH v6] Refactor part of the oom report in dump_header

2018-05-28 Thread Michal Hocko
On Sun 27-05-18 10:32:31, ufo19890607 wrote: > The dump_header does not print the memcg's name when the system > oom happened, so users cannot locate the certain container which > contains the task that has been killed by the oom killer. > > I follow the advices of David Rientjes a

Re: [PATCH] doc: document scope NOFS, NOIO APIs

2018-05-25 Thread Michal Hocko
On Fri 25-05-18 08:17:15, Dave Chinner wrote: > On Thu, May 24, 2018 at 01:43:41PM +0200, Michal Hocko wrote: [...] > > +FS/IO code then simply calls the appropriate save function right at the > > +layer where a lock taken from the reclaim context (e.g. shrinker) and > > +the

Re: [PATCH] doc: document scope NOFS, NOIO APIs

2018-05-25 Thread Michal Hocko
On Thu 24-05-18 14:52:02, Jonathan Corbet wrote: > On Thu, 24 May 2018 13:43:41 +0200 > Michal Hocko <mho...@kernel.org> wrote: > > > From: Michal Hocko <mho...@suse.com> > > > > Although the api is documented in the source code Ted has pointed out >

Re: [PATCH] doc: document scope NOFS, NOIO APIs

2018-05-25 Thread Michal Hocko
On Thu 24-05-18 09:37:18, Randy Dunlap wrote: > On 05/24/2018 04:43 AM, Michal Hocko wrote: [...] > > +The traditional way to avoid this deadlock problem is to clear __GFP_FS > > +resp. __GFP_IO (note the later implies clearing the first as well) in > >

Re: [rfc patch] mm, oom: fix unnecessary killing of additional processes

2018-05-25 Thread Michal Hocko
duces a per-mm reaping timeout, initially set > at 10s. It requires that the oom reaper's list becomes a properly linked > list so that other mm's may be reaped while waiting for an mm's timeout to > expire. No timeouts please! The proper way to handle this problem is to simply teach the oom reaper to handle mlocked areas. -- Michal Hocko SUSE Labs

Re: [RFC PATCH v2 00/12] get rid of GFP_ZONE_TABLE/BAD

2018-05-24 Thread Michal Hocko
On Thu 24-05-18 08:18:18, Matthew Wilcox wrote: > On Thu, May 24, 2018 at 02:23:23PM +0200, Michal Hocko wrote: > > > If we had eight ZONEs, we could offer: > > > > No, please no more zones. What we have is quite a maint. burden on its > > own. Ideally we sh

Re: [PATCH] doc: document scope NOFS, NOIO APIs

2018-05-24 Thread Michal Hocko
On Thu 24-05-18 07:33:39, Shakeel Butt wrote: > On Thu, May 24, 2018 at 4:43 AM, Michal Hocko <mho...@kernel.org> wrote: [...] > > +The traditional way to avoid this deadlock problem is to clear __GFP_FS > > +resp. __GFP_IO (note the later implies clearing the first as

Re: [PATCH v1 09/10] mm/memory_hotplug: teach offline_pages() to not try forever

2018-05-24 Thread Michal Hocko
migrate_range and distinguish transient from permanent failures. In general we shouldn't even get here for pages which are not migrateable... -- Michal Hocko SUSE Labs

Re: [PATCH v2 0/7] mm: pages for hugetlb's overcommit may be able to charge to memcg

2018-05-24 Thread Michal Hocko
On Thu 24-05-18 21:58:49, TSUKADA Koutaro wrote: > On 2018/05/24 17:20, Michal Hocko wrote: > > On Thu 24-05-18 13:39:59, TSUKADA Koutaro wrote: > >> On 2018/05/23 3:54, Michal Hocko wrote: > > [...] > >>> I am also quite confused why you keep distinguish

Re: [PATCH v1 00/10] mm: online/offline 4MB chunks controlled by device driver

2018-05-24 Thread Michal Hocko
here. Do I make at least some sense or I am completely missing your point? -- Michal Hocko SUSE Labs

Re: [RFC PATCH v2 00/12] get rid of GFP_ZONE_TABLE/BAD

2018-05-24 Thread Michal Hocko
On Wed 23-05-18 22:19:19, Matthew Wilcox wrote: > On Tue, May 22, 2018 at 08:37:28PM +0200, Michal Hocko wrote: > > So why is this any better than the current code. Sure I am not a great > > fan of GFP_ZONE_TABLE because of how it is incomprehensible but this > > doesn't look

Re: [External] Re: [RFC PATCH v2 00/12] get rid of GFP_ZONE_TABLE/BAD

2018-05-24 Thread Michal Hocko
On Wed 23-05-18 16:07:16, Huaisheng HS1 Ye wrote: > From: Michal Hocko [mailto:mho...@kernel.org] > Sent: Wednesday, May 23, 2018 2:37 AM > > > > On Mon 21-05-18 23:20:21, Huaisheng Ye wrote: > > > From: Huaisheng Ye <ye...@lenovo.com> > > > >

Re: [PATCH v1 00/10] mm: online/offline 4MB chunks controlled by device driver

2018-05-24 Thread Michal Hocko
On Thu 24-05-18 12:45:50, David Hildenbrand wrote: > On 24.05.2018 11:31, Michal Hocko wrote: > > On Thu 24-05-18 10:31:30, David Hildenbrand wrote: [...] > >> Allowing to unplug such small chunks is actually the interesting thing. > > > > Not really. The vmemm

[PATCH] doc: document scope NOFS, NOIO APIs

2018-05-24 Thread Michal Hocko
From: Michal Hocko <mho...@suse.com> Although the api is documented in the source code Ted has pointed out that there is no mention in the core-api Documentation and there are people looking there to find answers how to use a specific API. Cc: "Darrick J. Wong" <darrick.

Re: [PATCH] Revert "mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE"

2018-05-24 Thread Michal Hocko
pcp_init(zone); > > - /* > - * The size of the CMA area is unknown now so we need to > - * prepare the memory for the usemap at maximum. > - */ > - if (IS_ENABLED(CONFIG_CMA) && j == ZONE_MOVABLE && > - pgdat->node_spanned_pages) { > - movable_size = node_end_pfn - pgdat->node_start_pfn; > - } > - > - if (!size && !movable_size) > + if (!size) > continue; > > set_pageblock_order(); > - if (movable_size) { > - zone->zone_start_pfn = pgdat->node_start_pfn; > - zone->spanned_pages = movable_size; > - setup_usemap(pgdat, zone, > - pgdat->node_start_pfn, movable_size); > - init_currently_empty_zone(zone, > - pgdat->node_start_pfn, movable_size); > - } else { > - setup_usemap(pgdat, zone, zone_start_pfn, size); > - init_currently_empty_zone(zone, zone_start_pfn, size); > - } > + setup_usemap(pgdat, zone, zone_start_pfn, size); > + init_currently_empty_zone(zone, zone_start_pfn, size); > memmap_init(size, nid, j, zone_start_pfn); > } > } > @@ -7951,7 +7928,7 @@ void free_contig_range(unsigned long pfn, unsigned > nr_pages) > } > #endif > > -#if defined CONFIG_MEMORY_HOTPLUG || defined CONFIG_CMA > +#ifdef CONFIG_MEMORY_HOTPLUG > /* > * The zone indicated has a new number of managed_pages; batch sizes and > percpu > * page high values need to be recalulated. > -- > 2.7.4 > -- Michal Hocko SUSE Labs

Re: [PATCH 0/2] mm->owner to mm->memcg fixes

2018-05-24 Thread Michal Hocko
On Wed 23-05-18 14:46:43, Eric W. Biederman wrote: > Michal Hocko <mho...@kernel.org> writes: > > > On Thu 10-05-18 14:14:18, Michal Hocko wrote: > >> On Fri 04-05-18 12:26:03, Eric W. Biederman wrote: > >> > > >> > Andrew can you pick up th

Re: [PATCH v1 00/10] mm: online/offline 4MB chunks controlled by device driver

2018-05-24 Thread Michal Hocko
On Thu 24-05-18 10:31:30, David Hildenbrand wrote: > On 24.05.2018 09:53, Michal Hocko wrote: > > I've had some questions before and I am not sure they are fully covered. > > At least not in the cover letter (I didn't get much further yet) which > > should give us

Re: [PATCH v2 0/7] mm: pages for hugetlb's overcommit may be able to charge to memcg

2018-05-24 Thread Michal Hocko
will allow users to exhaust > the entire memory of the system. Of course, this can be prevented by the > hugetlb cgroup, but even if we set the limit for memcg and hugetlb cgroup > respectively, as I asked in the first mail(set limit to 10GB), the > control will not work. -- Michal Hocko SUSE Labs

Re: [PATCH v2 0/7] mm: pages for hugetlb's overcommit may be able to charge to memcg

2018-05-24 Thread Michal Hocko
On Thu 24-05-18 13:39:59, TSUKADA Koutaro wrote: > On 2018/05/23 3:54, Michal Hocko wrote: [...] > > I am also quite confused why you keep distinguishing surplus hugetlb > > pages from regular preallocated ones. Being a surplus page is an > > implementation detail that w

Re: [PATCH 2/2] mm: do not warn on offline nodes unless the specific node is explicitly requested

2018-05-24 Thread Michal Hocko
On Thu 24-05-18 08:52:14, Anshuman Khandual wrote: > On 05/23/2018 07:36 PM, Michal Hocko wrote: > > On Wed 23-05-18 19:15:51, Anshuman Khandual wrote: > >> On 05/23/2018 06:25 PM, Michal Hocko wrote: > >>> when adding memory to a node that is currently offline. >

Re: [PATCH v1 00/10] mm: online/offline 4MB chunks controlled by device driver

2018-05-24 Thread Michal Hocko
in the hypervisor). Please expand on the kdump part. That is really confusing because hotplug should simply not depend on kdump at all. Moreover why don't you simply mark those pages reserved and pull them out from the page allocator? -- Michal Hocko SUSE Labs

Re: [PATCH 2/2] mm: do not warn on offline nodes unless the specific node is explicitly requested

2018-05-23 Thread Michal Hocko
On Wed 23-05-18 19:15:51, Anshuman Khandual wrote: > On 05/23/2018 06:25 PM, Michal Hocko wrote: > > when adding memory to a node that is currently offline. > > > > The VM_WARN_ON is just too loud without a good reason. In this > > particular case we are doing &g

[PATCH 2/2] mm: do not warn on offline nodes unless the specific node is explicitly requested

2018-05-23 Thread Michal Hocko
From: Michal Hocko <mho...@suse.com> Oscar has noticed that we splat linux kernel: WARNING: CPU: 0 PID: 64 at ./include/linux/gfp.h:467 vmemmap_alloc_block+0x4e/0xc9 [...] linux kernel: CPU: 0 PID: 64 Comm: kworker/u4:1 Tainted: GW E 4.17.0-rc5-next-20180517-1-default+ #66

[PATCH 0/2] few memory hotplug fixes

2018-05-23 Thread Michal Hocko
entures.net [2] http://lkml.kernel.org/r/20180523080108.ga30...@techadventures.net Michal Hocko (2): mm, memory_hotplug: make has_unmovable_pages more robust mm: do not warn on offline nodes unless the specific node is explicitly requested Diffstat include/linux/gfp.h | 2 +- mm/p

[PATCH 1/2] mm, memory_hotplug: make has_unmovable_pages more robust

2018-05-23 Thread Michal Hocko
From: Michal Hocko <mho...@suse.com> Oscar has reported: : Due to an unfortunate setting with movablecore, memblocks containing bootmem : memory (pages marked by get_page_bootmem()) ended up marked in zone_movable. : So while trying to remove that memory, the system failed in do_migrate

Re: [PATCH] mm: save two stranding bit in gfp_mask

2018-05-23 Thread Michal Hocko
t; > :: The code at line 2585 was first introduced by commit > :: 1da177e4c3f41524e886b7f1b8a0c1fc7321cac2 Linux-2.6.12-rc2 > > :: TO: Linus Torvalds <torva...@ppc970.osdl.org> > :: CC: Linus Torvalds <torva...@ppc970.osdl.org> > > --- > 0-DAY k

Re: [PATCH v2 0/7] mm: pages for hugetlb's overcommit may be able to charge to memcg

2018-05-22 Thread Michal Hocko
ugetlb pages into the memcg mix. They just do not belong there. Try to look at previous discussions why it has been decided to have a separate hugetlb pages at all. I am also quite confused why you keep distinguishing surplus hugetlb pages from regular preallocated ones. Being a surplus page is an implementation detail that we use for an internal accounting rather than something to exhibit to the userspace even more than we do currently. Just look at what [sw]hould when you need to adjust accounting - e.g. due to the pool resize. Are you going to uncharge those surplus pages ffrom memcg to reflect their persistence? -- Michal Hocko SUSE Labs

Re: [RFC PATCH v2 00/12] get rid of GFP_ZONE_TABLE/BAD

2018-05-22 Thread Michal Hocko
-- > include/linux/highmem.h | 4 +- > mm/vmpressure.c | 2 +- > mm/zsmalloc.c| 4 +- > 12 files changed, 26 insertions(+), 103 deletions(-) > > -- > 1.8.3.1 > -- Michal Hocko SUSE Labs

Re: [PATCH v2 0/7] mm: pages for hugetlb's overcommit may be able to charge to memcg

2018-05-22 Thread Michal Hocko
ing it difficult to use. HugeTLBfs > may support multiple huge page sizes, and in such a special environment > there is a desire to use HugeTLBfs. Well, then I would argue that you shouldn't use 64kB pages for your setup or allow THP for smaller sizes. Really hugetlb pages are by no means a substitute here. -- Michal Hocko SUSE Labs

Re: [PATCH 0/2] mm->owner to mm->memcg fixes

2018-05-22 Thread Michal Hocko
On Thu 10-05-18 14:14:18, Michal Hocko wrote: > On Fri 04-05-18 12:26:03, Eric W. Biederman wrote: > > > > Andrew can you pick up these two fixes as well. > > > > These address the issues Michal Hocko and Oleg Nesterov noticed. > > I completely got lost in th

Re: [PATCH] mm/THP: use hugepage_vma_check() in khugepaged_enter_vma_merge()

2018-05-22 Thread Michal Hocko
pings */ > + > + if (!hugepage_vma_check(vma)) > return 0; > hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; > hend = vma->vm_end & HPAGE_PMD_MASK; > -- > 2.9.5 -- Michal Hocko SUSE Labs

Re: [PATCH v4] Print the memcg's name when system-wide OOM happened

2018-05-22 Thread Michal Hocko
index 8ba6cb88cf58..3e0b725fb877 100644 > --- a/mm/oom_kill.c > +++ b/mm/oom_kill.c > @@ -433,6 +433,7 @@ static void dump_header(struct oom_control *oc, struct > task_struct *p) > if (is_memcg_oom(oc)) > mem_cgroup_print_oom_info(oc->memcg, p); > else { > + mem_cgroup_print_oom_memcg_name(oc->memcg, p); > show_mem(SHOW_MEM_FILTER_NODES, oc->nodemask); > if (is_dump_unreclaim_slabs()) > dump_unreclaimable_slab(); > -- > 2.14.1 -- Michal Hocko SUSE Labs

Re: [PATCH] Add the memcg print oom info for system oom

2018-05-22 Thread Michal Hocko
On Mon 21-05-18 14:11:21, David Rientjes wrote: > On Thu, 17 May 2018, Michal Hocko wrote: > > > this is not 5 lines at all. We dump memcg stats for the whole oom memcg > > subtree. For your patch it would be the whole subtree of the memcg of > > the oom victim. With cgr

Re: [PATCH] MAINTAINERS: Change hugetlbfs maintainer and update files

2018-05-22 Thread Michal Hocko
ntry to include linux-mm mail list and > additional hugetlbfs related files. hugetlb.c and hugetlb.h are > not 100% hugetlbfs, but a majority of their content is hugetlbfs > related. > > Signed-off-by: Mike Kravetz <mike.krav...@oracle.com> Thanks a lot Mike! Ack

Re: [PATCH -mm] mm, huge page: Copy to access sub-page last when copy huge page

2018-05-18 Thread Michal Hocko
It is also adds quite some non-intuitive code. So is this worth? Does any _real_ workload benefits from the change? > include/linux/mm.h | 3 ++- > mm/huge_memory.c | 3 ++- > mm/memory.c| 43 +++ > 3 files changed, 43 insertions(+), 6

Re: [PATCH v2] Print the memcg's name when system-wide OOM happened

2018-05-18 Thread Michal Hocko
pr_cont(" killed as a result of limit of "); > show_mem(SHOW_MEM_FILTER_NODES, oc->nodemask); > if (is_dump_unreclaim_slabs()) > dump_unreclaimable_slab(); I bet this doesn't compile with CONFIG_MEMCG=n. You either ne

Re: [PATCH] Revert "mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE"

2018-05-17 Thread Michal Hocko
On Thu 17-05-18 20:13:35, Ville Syrjälä wrote: > On Thu, May 17, 2018 at 06:49:47PM +0200, Michal Hocko wrote: > > On Thu 17-05-18 16:58:32, Ville Syrjälä wrote: > > > On Thu, May 17, 2018 at 04:36:29PM +0300, Ville Syrjälä wrote: > > > > On Thu, May 17, 2018 at

Re: [PATCH] Revert "mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE"

2018-05-17 Thread Michal Hocko
On Thu 17-05-18 18:49:47, Michal Hocko wrote: > On Thu 17-05-18 16:58:32, Ville Syrjälä wrote: > > On Thu, May 17, 2018 at 04:36:29PM +0300, Ville Syrjälä wrote: > > > On Thu, May 17, 2018 at 03:21:09PM +0200, Michal Hocko wrote: > > > > On Thu 17-05-18

Re: [PATCH] Revert "mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE"

2018-05-17 Thread Michal Hocko
On Thu 17-05-18 16:58:32, Ville Syrjälä wrote: > On Thu, May 17, 2018 at 04:36:29PM +0300, Ville Syrjälä wrote: > > On Thu, May 17, 2018 at 03:21:09PM +0200, Michal Hocko wrote: > > > On Thu 17-05-18 15:59:59, Ville Syrjala wrote: > > > > From: Ville Syrjälä

<    5   6   7   8   9   10   11   12   13   14   >