Re: [PATCH 4/7] mm/hmm: properly handle migration pmd

2018-08-28 Thread Michal Hocko
D migration > is only enabled in x86_64 systems. > Other architectures should treat PMD migration entries as bad. How can we have a migration pmd entry when the migration is not supported? -- Michal Hocko SUSE Labs

Re: [PATCH] memory_hotplug: fix kernel_panic on offline page processing

2018-08-28 Thread Michal Hocko
[Fixup Pavel's email - the patch is here for your reference http://lkml.kernel.org/r/20180828090539.41491-1-zaslo...@linux.ibm.com] On Tue 28-08-18 13:25:43, Michal Hocko wrote: > On Tue 28-08-18 11:05:39, Mikhail Zaslonko wrote: > > Within show_valid_zones() the function test_pages_

Re: [PATCH] memory_hotplug: fix kernel_panic on offline page processing

2018-08-28 Thread Michal Hocko
are basically impossible to find during the review are the reason why I was not all that happy about d0dc12e86b31. It added a margninal improvement but opened a can of warms. On the other hand maybe we just had to open that can one day... Acked-by: Michal Hocko Thanks! > --- > drive

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-27 Thread Michal Hocko
> Fixes: 39dde65c9940 ("shared page table for hugetlb page") > Cc: sta...@vger.kernel.org > Signed-off-by: Mike Kravetz With the locking expectations for mmu_notifier_invalidate_range exlained I do not see any other issues. Acked-by: Michal H

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-27 Thread Michal Hocko
On Mon 27-08-18 09:46:33, Jerome Glisse wrote: > On Mon, Aug 27, 2018 at 09:46:45AM +0200, Michal Hocko wrote: > > On Fri 24-08-18 11:08:24, Mike Kravetz wrote: > > > On 08/24/2018 01:41 AM, Michal Hocko wrote: > > > > On Thu 23-08-18 13:59:16, Mike Kravetz wrote: >

[PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-08-27 Thread Michal Hocko
From: Michal Hocko Tetsuo Handa has reported that it is possible to bypass the short sleep for PF_WQ_WORKER threads which was introduced by commit 373ccbe5927034b5 ("mm, vmstat: allow WQ concurrency to discover memory reclaim doesn't make any progress") and lock up the sys

[PATCH 1/3] xen/gntdev: fix up blockable calls to mn_invl_range_start

2018-08-27 Thread Michal Hocko
From: Michal Hocko 93065ac753e4 ("mm, oom: distinguish blockable mode for mmu notifiers") has introduced blockable parameter to all mmu_notifiers and the notifier has to back off when called in !blockable case and it could block down the road. The above commit i

[PATCH 3/3] Revert "mm, mmu_notifier: annotate mmu notifiers with blockable invalidate callbacks"

2018-08-27 Thread Michal Hocko
From: Michal Hocko This reverts commit 5ff7091f5a2ca1b7b642ca0dbdede8f693a56926. MMU_INVALIDATE_DOES_NOT_BLOCK flags was the only one used and it is no longer needed since 93065ac753e4 ("mm, oom: distinguish blockable mode for mmu notifiers"). We now have a full support for per range

[PATCH 0/3] mmu_notifiers follow ups

2018-08-27 Thread Michal Hocko
Hi Andrew, Tetsuo has noticed some fallouts from 93065ac753e4 ("mm, oom: distinguish blockable mode for mmu notifiers"). One of them has been fixed and picked up by AMD/DRM maintainer [1]. XEN issue is fixed by patch 1. I have also clarified expectations about blockable semantic of

[PATCH 2/3] mm, mmu_notifier: be explicit about range invalition non-blocking mode

2018-08-27 Thread Michal Hocko
From: Michal Hocko If invalidate_range_start is called for !blocking mode then all callbacks have to guarantee they will no block/sleep. The same obviously applies to invalidate_range_end because this operation pairs with the former and they are called from the same context. Make sure

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-27 Thread Michal Hocko
On Fri 24-08-18 11:08:24, Mike Kravetz wrote: > On 08/24/2018 01:41 AM, Michal Hocko wrote: > > On Thu 23-08-18 13:59:16, Mike Kravetz wrote: > > > > Acked-by: Michal Hocko > > > > One nit below. > > > > [...] > >> diff --git a/mm/hugetlb.

Re: [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 14:18:44, Christian König wrote: > Am 24.08.2018 um 14:03 schrieb Michal Hocko: > > On Fri 24-08-18 13:57:52, Christian König wrote: > > > Am 24.08.2018 um 13:52 schrieb Michal Hocko: > > > > On Fri 24-08-18 13:43:16, Christian König wrote: > >

Re: [PATCH] x86/speculation/l1tf: suggest what to do on systems with too much RAM

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 14:10:54, Vlastimil Babka wrote: > On 08/24/2018 12:36 PM, Vlastimil Babka wrote: > > On 08/24/2018 09:32 AM, Vlastimil Babka wrote: > >> On 08/23/2018 09:27 PM, Michal Hocko wrote: > >>> On Thu 23-08-18 16:28:12, Vlastimil Babka wrote: >

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-24 Thread Michal Hocko
te_range in try_to_unmap_one, unless I am terribly confused. This would suggest 369ea8242c0fb is incorrect. -- Michal Hocko SUSE Labs

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-24 Thread Michal Hocko
erefore, > check for the possibility of PMD sharing before locking so that > notifiers can prepare for the worst possible case. > > Fixes: 39dde65c9940 ("shared page table for hugetlb page") > Cc: sta...@vger.kernel.org > Signed-off-by: Mike Kravetz Acked-by: Michal Hocko

Re: [PATCH 1/2] Revert "x86/e820: put !E820_TYPE_RAM regions into memblock.reserved"

2018-08-24 Thread Michal Hocko
/* > > -* all !E820_TYPE_RAM ranges (including gap ranges) are put > > -* into memblock.reserved to make sure that struct pages in > > -* such regions are not left uninitialized after bootup. > > -*/ > > if (entry->type != E820_TYPE_RAM && entry->type != > > E820_TYPE_RESERVED_KERN) > > - memblock_reserve(entry->addr, entry->size); > > - else > > - memblock_add(entry->addr, entry->size); > > + continue; > > + > > + memblock_add(entry->addr, entry->size); > > } > > > > /* Throw away partial pages: */ > > -- > > 2.18.0 > > > > -- Michal Hocko SUSE Labs

Re: [PATCH] mm/hugetlb: filter out hugetlb pages if HUGEPAGE migration is not supported.

2018-08-24 Thread Michal Hocko
sn't make any sense to return a pfn to non-migrateable huge page. > Acked-by: Michal Hocko > Reported-by: Haren Myneni > CC: Naoya Horiguchi > Signed-off-by: Aneesh Kumar K.V I would add Fixes: 72b39cfc4d75 ("mm, memory_hotplug: do not fail offlining too early") N

Re: [PATCH v2 1/3] mm: rework memcg kernel stack accounting

2018-08-24 Thread Michal Hocko
On Thu 23-08-18 09:23:50, Roman Gushchin wrote: > On Wed, Aug 22, 2018 at 04:12:13PM +0200, Michal Hocko wrote: [...] > > > @@ -248,9 +253,20 @@ static unsigned long *alloc_thread_stack_node(struct > > > task_struct *tsk, int node) > > > static inline void free_t

Re: [PATCH] x86/speculation/l1tf: suggest what to do on systems with too much RAM

2018-08-23 Thread Michal Hocko
without going to call for help. The message does tell them how to _enable_ it and point them to the documentation on how to _decide_. -- Michal Hocko SUSE Labs

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-23 Thread Michal Hocko
On Thu 23-08-18 10:56:30, Mike Kravetz wrote: > On 08/23/2018 10:01 AM, Mike Kravetz wrote: > > On 08/23/2018 05:48 AM, Michal Hocko wrote: > >> On Tue 21-08-18 18:10:42, Mike Kravetz wrote: > >> [...] > >> > >> OK, after burning myself when try

Re: [PATCH] x86/speculation/l1tf: suggest what to do on systems with too much RAM

2018-08-23 Thread Michal Hocko
?id=1105536 > > Suggested-by: Michal Hocko > Cc: sta...@vger.kernel.org > Signed-off-by: Vlastimil Babka I wouldn't bother with max_pfn-half_pa part but other than that this is much more useful than the original message. Acked-by: Michal Hocko > --- > arch/x86/kernel/cpu/bugs.c

Re: [PATCH] x86/speculation/l1tf: suggest what to do on systems with too much RAM

2018-08-23 Thread Michal Hocko
M bit > error > rate will be significant. L1TF is probably one of your smaller problems > in this case... There are people who care about L1TF mitigations. I am not going to question their motivation. In any case a hint how to make the mitigation active again sounds more useful than something that sounds as scary as "you are vulnerable". -- Michal Hocko SUSE Labs

Re: [PATCH] x86/speculation/l1tf: fix off-by-one error when warning that system has too much RAM

2018-08-23 Thread Michal Hocko
bugzilla.suse.com/show_bug.cgi?id=1105536 > > Reported-by: George Anchev > Reported-by: Christopher Snowhill > Fixes: 17dbca119312 ("x86/speculation/l1tf: Add sysfs reporting for l1tf") > Cc: sta...@vger.kernel.org > Signed-off-by: Vlastimil Babka Yes this should be l

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-23 Thread Michal Hocko
signed long a_start = check_addr & PUD_MASK; > + unsigned long a_end = a_start + PUD_SIZE; I guess this should be rather in HPAGE_SIZE * PTRS_PER_PTE units as huge_pmd_unshare does. -- Michal Hocko SUSE Labs

Re: [PATCH] fs: fix local var type

2018-08-23 Thread Michal Hocko
; const u8 *ptr = buf; > - int i, linelen, remaining = len; > + int i, linelen; > + size_t remaining = len; > char *buffer; > size_t size; > int ret; > -- > 2.7.4 > -- Michal Hocko SUSE Labs

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-23 Thread Michal Hocko
On Thu 23-08-18 11:21:12, Kirill A. Shutemov wrote: > On Thu, Aug 23, 2018 at 09:30:35AM +0200, Michal Hocko wrote: > > On Wed 22-08-18 09:48:16, Mike Kravetz wrote: > > > On 08/22/2018 05:28 AM, Michal Hocko wrote: > > > > On Tue 21-08-18 18:10:42, Mike Kravetz wrot

Re: [PATCH] mm: respect arch_dup_mmap() return value

2018-08-23 Thread Michal Hocko
n't have much time and I didn't have a high motivation because I simple disagreed with the patch. > Fixes: d70f2a14b72a4 ("include/linux/sched/mm.h: uninline mmdrop_async(), > etc") > Cc: Andrew Morton > Cc: sta...@vger.kernel.org > Signed-off-by: Nadav Amit Acked-by: Mic

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-23 Thread Michal Hocko
On Wed 22-08-18 09:48:16, Mike Kravetz wrote: > On 08/22/2018 05:28 AM, Michal Hocko wrote: > > On Tue 21-08-18 18:10:42, Mike Kravetz wrote: > > [...] > >> diff --git a/mm/rmap.c b/mm/rmap.c > >> index eb477809a5c0..8cf853a4b093 100644 > >> --- a/mm/rm

Re: [PATCH v2 1/3] mm: rework memcg kernel stack accounting

2018-08-22 Thread Michal Hocko
hich can't be released even by significant > memory pressure. > > As a cgroup structure can take a significant amount of memory > (first of all, per-cpu data like memcg statistics), it leads > to a noticeable waste of memory. > > Signed-off-by: Roman Gushchin > Cc: J

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-22 Thread Michal Hocko
if (huge_pmd_unshare(mm, , pvmw.pte)) { huge_pmd_unshare is documented to require a pte lock. Where do we take it? -- Michal Hocko SUSE Labs

Re: [PATCH v2 0/2] mm: soft-offline: fix race against page allocation

2018-08-22 Thread Michal Hocko
wed-by. > This fix should work on the reported issue, but rewriting soft-offlining > without PageHWPoison flag would be the better fix (no actual patch yet.) If we can go with the later the I would obviously prefer that. I cannot promise to work on the patch though. I can help with reviewing of course. If this is important enough that people are hitting the issue in normal workloads then sure, let's go with the simple fix and continue on top of that. -- Michal Hocko SUSE Labs

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-08-22 Thread Michal Hocko
On Wed 22-08-18 06:07:40, Tetsuo Handa wrote: > On 2018/08/03 15:16, Michal Hocko wrote: [...] > >> Now that Roman's cgroup aware OOM killer patchset will be dropped from > >> linux-next.git , > >> linux-next.git will get the sleeping point removed. Please send this

Re: [PATCH 2/2] memcg, oom: emit oom report when there is no eligible task

2018-08-21 Thread Michal Hocko
Do you plan to repost these two? They are quite deep in the email thread so they can easily fall through cracks. On Wed 08-08-18 18:17:37, Michal Hocko wrote: > On Wed 08-08-18 10:45:15, Johannes Weiner wrote: [...] > > >From bba01122f739b05a689dbf1eeeb4f0e07affd4e7 Mon Sep 17 0

Re: [PATCH] mm: Fix comment for NODEMASK_ALLOC

2018-08-21 Thread Michal Hocko
On Tue 21-08-18 14:30:24, Oscar Salvador wrote: > On Tue, Aug 21, 2018 at 02:17:34PM +0200, Michal Hocko wrote: > > We do have CONFIG_NODES_SHIFT=10 in our SLES kernels for quite some > > time (around SLE11-SP3 AFAICS). > > > > Anyway, isn't NODES_ALLOC over engineere

Re: [PATCH] mm: Fix comment for NODEMASK_ALLOC

2018-08-21 Thread Michal Hocko
functions are called in deep stacks. I haven't gone through all of them but a patch which checks them all and removes NODES_ALLOC would be quite nice IMHO. -- Michal Hocko SUSE Labs

Re: [PATCH] mm: introduce kvvirt_to_page() helper

2018-08-20 Thread Michal Hocko
On Mon 20-08-18 10:07:44, Matthew Wilcox wrote: > On Mon, Aug 20, 2018 at 06:24:06PM +0200, Michal Hocko wrote: > > On Mon 20-08-18 07:49:23, Matthew Wilcox wrote: > > > On Mon, Aug 20, 2018 at 04:41:16PM +0200, Michal Hocko wrote: > > > > On Sat 18-08-

Re: Crash in MM code in v4.4.y, v4.9.y with TRANSPARENT_HUGEPAGE enabled

2018-08-20 Thread Michal Hocko
On Mon 20-08-18 11:03:53, Andi Kleen wrote: > On Mon, Aug 20, 2018 at 06:29:38PM +0200, Michal Hocko wrote: > > On Fri 17-08-18 15:27:33, Guenter Roeck wrote: > > > Hi, > > > > > > the following crash is seen in v4.4.148, v4.4.149, v4.9.120, and v4.9.121 >

Re: Crash in MM code in v4.4.y, v4.9.y with TRANSPARENT_HUGEPAGE enabled

2018-08-20 Thread Michal Hocko
() macros"). I do not see it in stable 4.4 tree and it has been introduced much later in 4.14. This one gave us quite some headache because it is s easy to overlook. -- Michal Hocko SUSE Labs

Re: [PATCH] mm: introduce kvvirt_to_page() helper

2018-08-20 Thread Michal Hocko
On Mon 20-08-18 07:49:23, Matthew Wilcox wrote: > On Mon, Aug 20, 2018 at 04:41:16PM +0200, Michal Hocko wrote: > > On Sat 18-08-18 20:49:01, Li RongQing wrote: > > > The new helper returns address mapping page, which has several users > > > in individual subsystem, li

Re: [PATCH] mm: introduce kvvirt_to_page() helper

2018-08-20 Thread Michal Hocko
flush_dcache_page(kvvirt_to_page(start)); > } > smp_wmb(); > #endif > @@ -2508,7 +2501,7 @@ static int tpacket_fill_skb(struct packet_sock *po, > struct sk_buff *skb, > return -EFAULT; > } > > - page = pgv_to_page(data); > + page = kvvirt_to_page(data); > data += len; > flush_dcache_page(page); > get_page(page); > @@ -4385,7 +4378,7 @@ static int packet_mmap(struct file *file, struct socket > *sock, > int pg_num; > > for (pg_num = 0; pg_num < rb->pg_vec_pages; pg_num++) { > - page = pgv_to_page(kaddr); > + page = kvvirt_to_page(kaddr); > err = vm_insert_page(vma, start, page); > if (unlikely(err)) > goto out; > -- > 2.16.2 > -- Michal Hocko SUSE Labs

Re: [PATCH] x86/speculation/l1tf: fix overflow on l1tf_pfn_limit() on 32bit

2018-08-20 Thread Michal Hocko
On Mon 20-08-18 13:41:03, Vlastimil Babka wrote: > On 08/20/2018 12:49 PM, Michal Hocko wrote: > > On Mon 20-08-18 11:58:35, Vlastimil Babka wrote: > >> On 32bit PAE kernels on 64bit hardware with enough physical bits, > >> l1tf_pfn_limit() will overflow unsigned

Re: [PATCH] x86/speculation/l1tf: fix overflow on l1tf_pfn_limit() on 32bit

2018-08-20 Thread Michal Hocko
rnels, especially with large pages Do you have any reference? > I'll dig into this more today, but if you have any hints from testing/fixing > your own backports please share them. Well, we have seen some issues on pre 3.12 kernels due to misbackporting prot_none mitigations. PMD format is different in pre 4.4 kernels. Maybe this is a similar issue. -- Michal Hocko SUSE Labs

Re: [PATCH] x86/speculation/l1tf: fix overflow on l1tf_pfn_limit() on 32bit

2018-08-20 Thread Michal Hocko
er.kernel.org > Signed-off-by: Vlastimil Babka Looks good to me. I would probably use phys_addr_t which would be more descriptive but this is just minor thing. Acked-by: Michal Hocko > --- > arch/x86/include/asm/processor.h | 4 ++-- > arch/x86/mm/init.c | 4 ++-- >

Re: [PATCH] mm, page_alloc: actually ignore mempolicies for high priority allocations

2018-08-16 Thread Michal Hocko
would now ignore mempolicies after the initial attempts > fail - if the code worked as people thought it does. > > Link: http://lkml.kernel.org/r/20180612122624.8045-1-vba...@suse.cz > Signed-off-by: Vlastimil Babka > Cc: Mel Gorman > Cc: Michal Hocko > Cc: David Ri

Re: [PATCH] mm: introduce kvvirt_to_page() helper

2018-08-16 Thread Michal Hocko
ddr(addr)) > + return virt_to_page(addr); > + else > + return vmalloc_to_page(addr); > +} > + > extern void kvfree(const void *addr); > > static inline atomic_t *compound_mapcount_ptr(struct page *page) > -- > 2.16.2 > -- Michal Hocko SUSE Labs

Re: [RFC PATCH 1/2] mm: rework memcg kernel stack accounting

2018-08-16 Thread Michal Hocko
should really back off as soon as possible rather than rely on a future action because this is quite subtle and prone to unexpected behavior. -- Michal Hocko SUSE Labs

Re: [RFC PATCH 2/2] mm: drain memcg stocks on css offlining

2018-08-15 Thread Michal Hocko
this makes sense. > Signed-off-by: Roman Gushchin > Cc: Johannes Weiner > Cc: Michal Hocko > Cc: Konstantin Khlebnikov > Cc: Tejun Heo Acked-by: Michal Hocko > --- > mm/memcontrol.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/mm/memcontrol.c b/

Re: [RFC PATCH 1/2] mm: rework memcg kernel stack accounting

2018-08-15 Thread Michal Hocko
d97 ("fork: Optimize task creation by caching two thread stacks per CPU if CONFIG_VMAP_STACK=y") AFAICS > Cc: Johannes Weiner > Cc: Michal Hocko > Cc: Andy Lutomirski > Cc: Konstantin Khlebnikov > Cc: Tejun Heo Yes this is t

Re: [PATCH v3] resource: Merge resources on a node when hot-adding memory

2018-08-10 Thread Michal Hocko
system. A report from Oscar shows that this is not the case though. -- Michal Hocko SUSE Labs

Re: [RFC v7 PATCH 4/4] mm: unmap special vmas with regular do_munmap()

2018-08-10 Thread Michal Hocko
ch of the special case should have its own patch and changelog explaining why it is safe. -- Michal Hocko SUSE Labs

Re: [PATCH 0/3] introduce memory.oom.group

2018-08-10 Thread Michal Hocko
On Thu 09-08-18 13:10:10, David Rientjes wrote: > On Wed, 8 Aug 2018, Michal Hocko wrote: > > > > > > In a cgroup-aware oom killer world, yes, we need the ability to > > > > > specify > > > > > that the usage of the entire subtree should b

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-09 Thread Michal Hocko
On Thu 09-08-18 10:27:09, Jerome Glisse wrote: > On Thu, Aug 09, 2018 at 10:24:15AM +0200, Michal Hocko wrote: > > On Wed 08-08-18 12:58:15, Jerome Glisse wrote: > > > On Wed, Aug 08, 2018 at 08:47:58AM +0200, Michal Hocko wrote: > > > > On Tue 07-08-18

Re: WARNING in try_charge

2018-08-09 Thread Michal Hocko
ing next OOM victim. > Therefore, this patch changes task_will_free_mem(current) to ignore > MMF_OOM_SKIP unless ALLOC_OOM allocation failed. And the patch is simply wrong for memcg. -- Michal Hocko SUSE Labs

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-09 Thread Michal Hocko
On Wed 08-08-18 12:58:15, Jerome Glisse wrote: > On Wed, Aug 08, 2018 at 08:47:58AM +0200, Michal Hocko wrote: > > On Tue 07-08-18 11:18:10, Jerome Glisse wrote: > > > On Tue, Aug 07, 2018 at 04:59:00PM +0200, Michal Hocko wrote: > > > > On Tue 07-08-18

Re: [PATCH RFC v2 02/10] mm: Make shrink_slab() lockless

2018-08-09 Thread Michal Hocko
An obvious question. Why didn't you go that way? What are pros/cons of both approaches? -- Michal Hocko SUSE Labs

Re: [PATCH 2/2] memcg, oom: emit oom report when there is no eligible task

2018-08-08 Thread Michal Hocko
On Wed 08-08-18 10:45:15, Johannes Weiner wrote: > On Wed, Aug 08, 2018 at 09:13:01AM +0200, Michal Hocko wrote: > > From: Michal Hocko > > > > Johannes had doubts that the current WARN in the memcg oom path > > when there is no eligible task is not all that useful be

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-08 Thread Michal Hocko
do not go tangent again. I really do not know how to send this simply message to you. I have tried so many times before. -- Michal Hocko SUSE Labs

Re: [PATCH 0/3] introduce memory.oom.group

2018-08-08 Thread Michal Hocko
e want the ability to > consider user-controlled subtrees as a single entity for comparison with > other user subtrees to select which subtree to target. This does not > imply that users want their entire subtree oom killed. Yeah, that's why oom.group == 0, no? Anyway, can we separate this discussion from the current series please? We are getting more and more tangent. Or do you still see the current state to be not mergeable? -- Michal Hocko SUSE Labs

Re: [PATCH RFC 01/10] rcu: Make CONFIG_SRCU unconditionally enabled

2018-08-08 Thread Michal Hocko
[CC Josh - the whole series is http://lkml.kernel.org/r/153365347929.19074.12509495712735843805.stgit@localhost.localdomain] On Wed 08-08-18 13:17:44, Kirill Tkhai wrote: > On 08.08.2018 10:20, Michal Hocko wrote: > > On Tue 07-08-18 18:37:36, Kirill Tkhai wrote: > >> Th

Re: [PATCH RFC 01/10] rcu: Make CONFIG_SRCU unconditionally enabled

2018-08-08 Thread Michal Hocko
can't we make this depend on MMU. Is anybody else than the reclaim asking for unconditional SRCU usage? Btw. I totaly agree with Steven. This is a very poor changelog. It is trivial to see what the patch does but it is far from clear why it is doing that and why we cannot go other ways. -- Michal

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-08 Thread Michal Hocko
On Wed 08-08-18 08:44:14, Michal Hocko wrote: > On Tue 07-08-18 16:54:25, Johannes Weiner wrote: [...] > > What the global OOM killer does in that situation is dump the header > > anyway: > > > > /* Found nothing?!?! Either we hang forever, or we panic.

[PATCH 2/2] memcg, oom: emit oom report when there is no eligible task

2018-08-08 Thread Michal Hocko
From: Michal Hocko Johannes had doubts that the current WARN in the memcg oom path when there is no eligible task is not all that useful because it doesn't really give any useful insight into the memcg state. My original intention was to make this lightweight but it is true that seeing a stack

[PATCH 1/2] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-08 Thread Michal Hocko
From: Michal Hocko "memcg, oom: move out_of_memory back to the charge path" has added a warning triggered when the oom killer cannot find any eligible task and so there is no way to reclaim the oom memcg under its hard limit. Further charges for such a memcg are forced and therefor

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread Michal Hocko
On Tue 07-08-18 11:18:10, Jerome Glisse wrote: > On Tue, Aug 07, 2018 at 04:59:00PM +0200, Michal Hocko wrote: > > On Tue 07-08-18 09:52:21, Jerome Glisse wrote: > > > On Tue, Aug 07, 2018 at 03:37:56PM +0200, osalva...@techadventures.net > > > wrote: &

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-08 Thread Michal Hocko
On Tue 07-08-18 16:54:25, Johannes Weiner wrote: > On Tue, Aug 07, 2018 at 10:23:32PM +0200, Michal Hocko wrote: > > On Tue 07-08-18 16:02:47, Johannes Weiner wrote: > > > On Tue, Aug 07, 2018 at 09:25:53AM +0200, Michal Hocko wrote: > > > > From: Michal Hocko >

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-07 Thread Michal Hocko
On Tue 07-08-18 16:02:47, Johannes Weiner wrote: > On Tue, Aug 07, 2018 at 09:25:53AM +0200, Michal Hocko wrote: > > From: Michal Hocko > > > > "memcg, oom: move out_of_memory back to the charge path" has added a > > warning triggered when the oom killer can

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-07 Thread Michal Hocko
cessarily imply altmap. So > with the above changes you change the expected behavior. Could you be more specific what is the expected behavior here? Is this about calling release_mem_region_adjustable? Why does is it not suitable for zone device ranges? -- Michal Hocko SUSE Labs

Re: WARNING in try_charge

2018-08-07 Thread Michal Hocko
he way how your syzbot automation works made it so much easier so thanks a lot! The patch has been posted http://lkml.kernel.org/r/20180807072553.14941-1-mho...@kernel.org -- Michal Hocko SUSE Labs

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-07 Thread Michal Hocko
is quite tricky and I am really reluctant to make it even more so without seeing this is really hurting real users with real workloads. -- Michal Hocko SUSE Labs

[PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-07 Thread Michal Hocko
From: Michal Hocko "memcg, oom: move out_of_memory back to the charge path" has added a warning triggered when the oom killer cannot find any eligible task and so there is no way to reclaim the oom memcg under its hard limit. Further charges for such a memcg are forced and therefor

Re: [RFC v6 PATCH 2/2] mm: mmap: zap pages with read mmap_sem in munmap

2018-08-06 Thread Michal Hocko
On Mon 06-08-18 15:19:06, Yang Shi wrote: > > > On 8/6/18 1:52 PM, Michal Hocko wrote: > > On Mon 06-08-18 13:48:35, Yang Shi wrote: > > > > > > On 8/6/18 1:41 PM, Michal Hocko wrote: > > > > On Mon 06-08-18 09:46:30, Yang Shi wrote: &g

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
On Tue 07-08-18 05:46:04, Tetsuo Handa wrote: > On 2018/08/07 5:34, Michal Hocko wrote: > > On Tue 07-08-18 05:26:23, Tetsuo Handa wrote: > >> On 2018/08/07 2:56, Michal Hocko wrote: > >>> So the oom victim indeed passed the above force path after the oom >

Re: [RFC v6 PATCH 2/2] mm: mmap: zap pages with read mmap_sem in munmap

2018-08-06 Thread Michal Hocko
On Mon 06-08-18 13:48:35, Yang Shi wrote: > > > On 8/6/18 1:41 PM, Michal Hocko wrote: > > On Mon 06-08-18 09:46:30, Yang Shi wrote: > > > > > > On 8/6/18 2:40 AM, Michal Hocko wrote: > > > > On Fri 03-08-18 14:01:58, Yang Shi wrote: &g

Re: [RFC v6 PATCH 2/2] mm: mmap: zap pages with read mmap_sem in munmap

2018-08-06 Thread Michal Hocko
On Mon 06-08-18 09:46:30, Yang Shi wrote: > > > On 8/6/18 2:40 AM, Michal Hocko wrote: > > On Fri 03-08-18 14:01:58, Yang Shi wrote: > > > > > > On 8/3/18 2:07 AM, Michal Hocko wrote: > > > > On Fri 27-07-18 02:10:14, Yang Shi wrote: [...] &g

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
On Tue 07-08-18 05:26:23, Tetsuo Handa wrote: > On 2018/08/07 2:56, Michal Hocko wrote: > > So the oom victim indeed passed the above force path after the oom > > invocation. But later on hit the page fault path and that behaved > > differently and for some reason the force

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
On Mon 06-08-18 21:45:53, Michal Hocko wrote: > [CCing Greg - the email thread starts here > http://lkml.kernel.org/r/5e979605729c1...@google.com] now for real > > On Mon 06-08-18 12:12:02, syzbot wrote: > > Hello, > > > > syzbot has tested the propose

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
dbf5668ed26 Mon Sep 17 00:00:00 2001 From: Michal Hocko Date: Mon, 6 Aug 2018 21:21:24 +0200 Subject: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task "memcg, oom: move out_of_memory back to the charge path" has added a warning triggered when the oom killer

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
; WARN(1,"Memory cgroup charge failed because of no reclaimable memory! " -- Michal Hocko SUSE Labs

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
On Mon 06-08-18 20:13:39, Michal Hocko wrote: > I simply do not see how this is possible. Let's try with the following > extended debugging patch. > > #syz test: git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git > 116b181bb646afedd770985de20a68721bdb2648 >

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
oom victim now\n", + current->comm, current->pid); } /** -- Michal Hocko SUSE Labs

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
On Mon 06-08-18 19:44:10, Michal Hocko wrote: > On Mon 06-08-18 08:42:02, syzbot wrote: > > Hello, > > > > syzbot has tested the proposed patch but the reproducer still triggered > > crash: > > WARNING in try_charge > > > > Killed process 6410 (syz

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
current->flags & PF_EXITING)) goto force; -- Michal Hocko SUSE Labs

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
On Mon 06-08-18 16:58:01, Dmitry Vyukov wrote: > On Mon, Aug 6, 2018 at 4:21 PM, Michal Hocko wrote: > > On Mon 06-08-18 13:57:38, Dmitry Vyukov wrote: > >> On Mon, Aug 6, 2018 at 1:02 PM, Michal Hocko wrote: > > [...] > >> >> A much > >> >&

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
On Mon 06-08-18 23:41:22, Tetsuo Handa wrote: > +David Howells > > On 2018/08/06 20:32, Michal Hocko wrote: > > On Mon 06-08-18 04:27:02, syzbot wrote: > >> Hello, > >> > >> syzbot has tested the proposed patch and the reproducer did not trigger

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
On Mon 06-08-18 13:57:38, Dmitry Vyukov wrote: > On Mon, Aug 6, 2018 at 1:02 PM, Michal Hocko wrote: [...] > >> A much > >> friendlier for user way to say this would be print a message at the > >> point of misconfiguration saying what exactly is wrong, e.g. "

Re: [RFC v6 PATCH 1/2] mm: refactor do_munmap() to extract the common part

2018-08-06 Thread Michal Hocko
On Fri 03-08-18 13:47:19, Yang Shi wrote: > > > On 8/3/18 1:53 AM, Michal Hocko wrote: > > On Fri 27-07-18 02:10:13, Yang Shi wrote: > > > Introduces three new helper functions: > > >* munmap_addr_sanity() > > >* munma

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
240 > > Note: testing is done by a robot and is best-effort only. OK, so this smells like a problem in the previous group oom changes. Or maybe it is not very easy to reproduce? -- Michal Hocko SUSE Labs

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
On Mon 06-08-18 19:47:00, Tetsuo Handa wrote: > On 2018/08/06 19:39, Dmitry Vyukov wrote: > > On Mon, Aug 6, 2018 at 11:48 AM, Michal Hocko wrote: > >> Btw. running with the above diff on top might help us to ideantify > >> whether this is a pre-mature warning o

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
On Mon 06-08-18 12:34:30, Dmitry Vyukov wrote: > On Mon, Aug 6, 2018 at 11:48 AM, Michal Hocko wrote: > > On Mon 06-08-18 11:30:37, Dmitry Vyukov wrote: > >> On Mon, Aug 6, 2018 at 11:15 AM, Michal Hocko wrote: > > [...] > >> > More interesti

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
On Mon 06-08-18 11:30:37, Dmitry Vyukov wrote: > On Mon, Aug 6, 2018 at 11:15 AM, Michal Hocko wrote: [...] > > More interesting stuff is higher in the kernel log > > : [ 366.435015] > > oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=/,mems_allowed=0,oom_m

Re: [RFC v6 PATCH 2/2] mm: mmap: zap pages with read mmap_sem in munmap

2018-08-06 Thread Michal Hocko
On Fri 03-08-18 14:01:58, Yang Shi wrote: > > > On 8/3/18 2:07 AM, Michal Hocko wrote: > > On Fri 27-07-18 02:10:14, Yang Shi wrote: > > > When running some mmap/munmap scalability tests with large memory (i.e. > > > > 300GB), the below hung task issue may h

Re: WARNING in try_charge

2018-08-06 Thread Michal Hocko
warning is exactly to call that out. If this is some sort of race and we warn too eagerly is still to be checked and potentially fixed of course. -- Michal Hocko SUSE Labs

Re: [LKP] [mm, oom] c1e4c54f9c: BUG:KASAN:null-ptr-deref_in_d

2018-08-06 Thread Michal Hocko
id, > + from_kuid(_user_ns, task_uid(p))); > + pr_cont("\n"); > if (is_memcg_oom(oc)) > mem_cgroup_print_oom_meminfo(oc->memcg); > > Thanks > Zhoujian -- Michal Hocko SUSE Labs

Re: [RFC v6 PATCH 2/2] mm: mmap: zap pages with read mmap_sem in munmap

2018-08-03 Thread Michal Hocko
because munmap holds mmap_sem exclusively from very beginning to > all the way down to the end, and doesn't release it in the middle. When > unmapping large mapping, it may take long time (take ~18 seconds to > unmap 320GB mapping with every single page mapped on an idle machine). > > Zapping

Re: [RFC v6 PATCH 1/2] mm: refactor do_munmap() to extract the common part

2018-08-03 Thread Michal Hocko
error = __split_vma(mm, vma, start, 0); > + error = __split_vma(mm, tmp, start, 0); > if (error) > return error; > - prev = vma; > + *prev = tmp; > } > > /* Does it split the last one? */ > @@ -2747,7 +2758,48 @@ int do_munmap(struct mm_struct *mm, unsigned long > start, size_t len, > if (error) > return error; > } > - vma = prev ? prev->vm_next : mm->mmap; > + > + *vma = *prev ? (*prev)->vm_next : mm->mmap; > + > + return 1; > +} the patch would be much more easier to read if you didn't do vma->tmp renaming. -- Michal Hocko SUSE Labs

Re: [PATCH v1] mm:memcg: skip memcg of current in mem_cgroup_soft_limit_reclaim

2018-08-03 Thread Michal Hocko
On Fri 03-08-18 14:59:34, Zhaoyang Huang wrote: > On Fri, Aug 3, 2018 at 2:18 PM Michal Hocko wrote: > > > > On Fri 03-08-18 14:11:26, Zhaoyang Huang wrote: > > > On Fri, Aug 3, 2018 at 1:48 PM Zhaoyang Huang > > > wrote: > > > > > > >

Re: [RFC 2/2] mm: harden alloc_pages code paths against bogus nodes

2018-08-03 Thread Michal Hocko
On Thu 02-08-18 22:17:49, Jeremy Linton wrote: > Hi, > > On 08/02/2018 02:31 AM, Michal Hocko wrote: > > On Wed 01-08-18 15:04:18, Jeremy Linton wrote: > > > Its possible to crash __alloc_pages_nodemask by passing it > > > bogus node ids. This is cau

Re: [RFC 1/2] slub: Avoid trying to allocate memory on offline nodes

2018-08-03 Thread Michal Hocko
On Thu 02-08-18 22:21:53, Jeremy Linton wrote: > Hi, > > On 08/02/2018 04:15 AM, Michal Hocko wrote: > > On Wed 01-08-18 15:04:17, Jeremy Linton wrote: > > [...] > > > @@ -2519,6 +2519,8 @@ static void *___slab_alloc(struct kmem_cache *s,

Re: [PATCH v1] mm:memcg: skip memcg of current in mem_cgroup_soft_limit_reclaim

2018-08-03 Thread Michal Hocko
jor fault during the test. I have asked already asked. Why do you use the soft limit in the first place? It is known to cause excessive reclaim and long stalls. -- Michal Hocko SUSE Labs

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-08-03 Thread Michal Hocko
On Fri 03-08-18 07:05:54, Tetsuo Handa wrote: > On 2018/07/31 14:09, Michal Hocko wrote: > > On Tue 31-07-18 06:01:48, Tetsuo Handa wrote: > >> On 2018/07/31 4:10, Michal Hocko wrote: > >>> Since should_reclaim_retry() should be a natural reschedule point,

<    1   2   3   4   5   6   7   8   9   10   >