Re: [PATCH] mm: memcontrol: print proper OOM header when no eligible victim left

2018-09-08 Thread Tetsuo Handa
On 2018/09/08 22:57, Johannes Weiner wrote: > On Sat, Sep 08, 2018 at 10:36:06PM +0900, Tetsuo Handa wrote: >> Due to commit d75da004c708c9fc ("oom: improve oom disable handling") and >> commit 3100dab2aa09dc6e ("mm: memcontrol: print proper OOM header when >

Re: [PATCH] mm: memcontrol: print proper OOM header when no eligible victim left

2018-09-08 Thread Tetsuo Handa
On 2018/09/08 22:57, Johannes Weiner wrote: > On Sat, Sep 08, 2018 at 10:36:06PM +0900, Tetsuo Handa wrote: >> Due to commit d75da004c708c9fc ("oom: improve oom disable handling") and >> commit 3100dab2aa09dc6e ("mm: memcontrol: print proper OOM header when >

Re: [PATCH] mm: memcontrol: print proper OOM header when no eligible victim left

2018-09-08 Thread Tetsuo Handa
+--- > 2 files changed, 10 insertions(+), 5 deletions(-) > Now that above patch went to 4.19-rc3, please apply below one. >From eb2bff2ed308da04785bcf541dd3f748286bfa23 Mon Sep 17 00:00:00 2001 From: Tetsuo Handa Date: Sat, 8 Sep 2018 22:26:28 +0900 Subject: [PATCH] mm, oom: Don't emi

Re: [PATCH] mm: memcontrol: print proper OOM header when no eligible victim left

2018-09-08 Thread Tetsuo Handa
+--- > 2 files changed, 10 insertions(+), 5 deletions(-) > Now that above patch went to 4.19-rc3, please apply below one. >From eb2bff2ed308da04785bcf541dd3f748286bfa23 Mon Sep 17 00:00:00 2001 From: Tetsuo Handa Date: Sat, 8 Sep 2018 22:26:28 +0900 Subject: [PATCH] mm, oom: Don't emi

Re: [PATCH 3/6] selinux: convert to kvmalloc

2018-09-07 Thread Tetsuo Handa
On 2018/09/08 1:56, Kent Overstreet wrote: > @@ -329,8 +328,7 @@ int avtab_alloc(struct avtab *h, u32 nrules) > nslot = MAX_AVTAB_HASH_BUCKETS; > mask = nslot - 1; > > - h->htable = flex_array_alloc(sizeof(struct avtab_node *), nslot, > -

Re: [PATCH 3/6] selinux: convert to kvmalloc

2018-09-07 Thread Tetsuo Handa
On 2018/09/08 1:56, Kent Overstreet wrote: > @@ -329,8 +328,7 @@ int avtab_alloc(struct avtab *h, u32 nrules) > nslot = MAX_AVTAB_HASH_BUCKETS; > mask = nslot - 1; > > - h->htable = flex_array_alloc(sizeof(struct avtab_node *), nslot, > -

Re: BUG: bad usercopy in __check_object_size (2)

2018-09-07 Thread Tetsuo Handa
On 2018/09/08 0:29, syzbot wrote: > syzbot has found a reproducer for the following crash on: > > HEAD commit:    28619527b8a7 Merge git://git.kernel.org/pub/scm/linux/kern.. > git tree:   bpf > console output: https://syzkaller.appspot.com/x/log.txt?x=124e64d140 > kernel config: 

Re: BUG: bad usercopy in __check_object_size (2)

2018-09-07 Thread Tetsuo Handa
On 2018/09/08 0:29, syzbot wrote: > syzbot has found a reproducer for the following crash on: > > HEAD commit:    28619527b8a7 Merge git://git.kernel.org/pub/scm/linux/kern.. > git tree:   bpf > console output: https://syzkaller.appspot.com/x/log.txt?x=124e64d140 > kernel config: 

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-09-06 Thread Tetsuo Handa
Tetsuo Handa wrote: > Michal Hocko wrote: > > > I assert that we should fix af5679fbc669f31f. > > > > If you can come up with reasonable patch which doesn't complicate the > > code and it is a clear win for both this particular workload as well as > > other

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-09-06 Thread Tetsuo Handa
Tetsuo Handa wrote: > Michal Hocko wrote: > > > I assert that we should fix af5679fbc669f31f. > > > > If you can come up with reasonable patch which doesn't complicate the > > code and it is a clear win for both this particular workload as well as > > other

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-09-06 Thread Tetsuo Handa
Michal Hocko wrote: > > I assert that we should fix af5679fbc669f31f. > > If you can come up with reasonable patch which doesn't complicate the > code and it is a clear win for both this particular workload as well as > others then why not. Why can't we do "at least MMF_OOM_SKIP should be set

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-09-06 Thread Tetsuo Handa
Michal Hocko wrote: > > I assert that we should fix af5679fbc669f31f. > > If you can come up with reasonable patch which doesn't complicate the > code and it is a clear win for both this particular workload as well as > others then why not. Why can't we do "at least MMF_OOM_SKIP should be set

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-09-05 Thread Tetsuo Handa
Michal Hocko wrote: > On Wed 05-09-18 22:53:33, Tetsuo Handa wrote: > > On 2018/09/05 22:40, Michal Hocko wrote: > > > Changelog said > > > > > > "Although this is possible in principle let's wait for it to actually > > > happen in real

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-09-05 Thread Tetsuo Handa
Michal Hocko wrote: > On Wed 05-09-18 22:53:33, Tetsuo Handa wrote: > > On 2018/09/05 22:40, Michal Hocko wrote: > > > Changelog said > > > > > > "Although this is possible in principle let's wait for it to actually > > > happen in real

Re: INFO: task hung in vfat_lookup

2018-09-05 Thread Tetsuo Handa
On 2018/09/05 20:19, syzbot wrote: > Hello, > > syzbot found the following crash on: > > HEAD commit:    420f51f4ab6b Merge tag 'arm64-fixes' of git://git.kernel.o.. > git tree:   upstream > console output: https://syzkaller.appspot.com/x/log.txt?x=11296c9240 > kernel config: 

Re: INFO: task hung in vfat_lookup

2018-09-05 Thread Tetsuo Handa
On 2018/09/05 20:19, syzbot wrote: > Hello, > > syzbot found the following crash on: > > HEAD commit:    420f51f4ab6b Merge tag 'arm64-fixes' of git://git.kernel.o.. > git tree:   upstream > console output: https://syzkaller.appspot.com/x/log.txt?x=11296c9240 > kernel config: 

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-09-05 Thread Tetsuo Handa
On 2018/09/05 22:40, Michal Hocko wrote: > Changelog said > > "Although this is possible in principle let's wait for it to actually > happen in real life before we make the locking more complex again." > > So what is the real life workload that hits it? The log you have pasted > below doesn't

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-09-05 Thread Tetsuo Handa
On 2018/09/05 22:40, Michal Hocko wrote: > Changelog said > > "Although this is possible in principle let's wait for it to actually > happen in real life before we make the locking more complex again." > > So what is the real life workload that hits it? The log you have pasted > below doesn't

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-09-05 Thread Tetsuo Handa
On 2018/08/24 9:31, Tetsuo Handa wrote: > For now, I don't think we need to add af5679fbc669f31f to the list for > CVE-2016-10723, for af5679fbc669f31f might cause premature next OOM victim > selection (especially with CONFIG_PREEMPT=y kernels) due to > >__allo

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-09-05 Thread Tetsuo Handa
On 2018/08/24 9:31, Tetsuo Handa wrote: > For now, I don't think we need to add af5679fbc669f31f to the list for > CVE-2016-10723, for af5679fbc669f31f might cause premature next OOM victim > selection (especially with CONFIG_PREEMPT=y kernels) due to > >__allo

Re: [PATCH] security: tomoyo: Fix obsolete function

2018-09-04 Thread Tetsuo Handa
On 2018/09/04 17:41, Ding Xiang wrote: > simple_strtoul is obsolete, and use kstrtouint instead > > Signed-off-by: Ding Xiang Acked-by: Tetsuo Handa > --- > security/tomoyo/common.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/se

Re: [PATCH] security: tomoyo: Fix obsolete function

2018-09-04 Thread Tetsuo Handa
On 2018/09/04 17:41, Ding Xiang wrote: > simple_strtoul is obsolete, and use kstrtouint instead > > Signed-off-by: Ding Xiang Acked-by: Tetsuo Handa > --- > security/tomoyo/common.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/se

[PATCH] kernel/{lockdep,hung_task}: Show locks and backtrace of running tasks.

2018-09-03 Thread Tetsuo Handa
at calling lockdep_print_held_locks() on a running thread is considered unsafe, it is useful for syzbot to show locks and backtrace of running tasks. Thus, let's allow it if CONFIG_DEBUG_AID_FOR_SYZBOT is defined. [1] https://syzkaller.appspot.com/bug?id=8bab7a6a5597bb10f90e8227a7d8a483748d93be Signed-off-by: Te

[PATCH] kernel/{lockdep,hung_task}: Show locks and backtrace of running tasks.

2018-09-03 Thread Tetsuo Handa
at calling lockdep_print_held_locks() on a running thread is considered unsafe, it is useful for syzbot to show locks and backtrace of running tasks. Thus, let's allow it if CONFIG_DEBUG_AID_FOR_SYZBOT is defined. [1] https://syzkaller.appspot.com/bug?id=8bab7a6a5597bb10f90e8227a7d8a483748d93be Signed-off-by: Te

Re: [PATCH 0/8] CaitSith LSM module

2018-09-01 Thread Tetsuo Handa
On 2017/10/22 2:17, Casey Schaufler wrote: >> As one year elapsed since I proposed CaitSith for upstream, I'd like to >> hear the status again. I looked at >> http://schd.ws/hosted_files/lss2017/8b/201709-LinuxSecuritySummit-Stacking.pdf >> . >> How is ETA for Security Module Stacking? Is it a

Re: [PATCH 0/8] CaitSith LSM module

2018-09-01 Thread Tetsuo Handa
On 2017/10/22 2:17, Casey Schaufler wrote: >> As one year elapsed since I proposed CaitSith for upstream, I'd like to >> hear the status again. I looked at >> http://schd.ws/hosted_files/lss2017/8b/201709-LinuxSecuritySummit-Stacking.pdf >> . >> How is ETA for Security Module Stacking? Is it a

Re: [PATCH 2/4] tty: Hold tty_ldisc_lock() during tty_reopen()

2018-08-31 Thread Tetsuo Handa
On 2018/08/31 15:51, Jiri Slaby wrote: > On 08/29/2018, 05:19 PM, Tetsuo Handa wrote: >> On 2018/08/29 11:23, Dmitry Safonov wrote: >>> tty_ldisc_reinit() doesn't race with neither tty_ldisc_hangup() >>> nor set_ldisc() nor tty_ldisc_release() as they use tty lock. >&

Re: [PATCH 2/4] tty: Hold tty_ldisc_lock() during tty_reopen()

2018-08-31 Thread Tetsuo Handa
On 2018/08/31 15:51, Jiri Slaby wrote: > On 08/29/2018, 05:19 PM, Tetsuo Handa wrote: >> On 2018/08/29 11:23, Dmitry Safonov wrote: >>> tty_ldisc_reinit() doesn't race with neither tty_ldisc_hangup() >>> nor set_ldisc() nor tty_ldisc_release() as they use tty lock. >&

Re: [PATCH 2/4] tty: Hold tty_ldisc_lock() during tty_reopen()

2018-08-29 Thread Tetsuo Handa
On 2018/08/29 11:23, Dmitry Safonov wrote: > tty_ldisc_reinit() doesn't race with neither tty_ldisc_hangup() > nor set_ldisc() nor tty_ldisc_release() as they use tty lock. > But it races with anyone who expects line discipline to be the same > after hoding read semaphore in tty_ldisc_ref(). > >

Re: [PATCH 2/4] tty: Hold tty_ldisc_lock() during tty_reopen()

2018-08-29 Thread Tetsuo Handa
On 2018/08/29 11:23, Dmitry Safonov wrote: > tty_ldisc_reinit() doesn't race with neither tty_ldisc_hangup() > nor set_ldisc() nor tty_ldisc_release() as they use tty lock. > But it races with anyone who expects line discipline to be the same > after hoding read semaphore in tty_ldisc_ref(). > >

Re: [PATCH v2] n_tty: Protect tty->disc_data using refcount.

2018-08-29 Thread Tetsuo Handa
On 2018/08/29 22:53, Jiri Slaby wrote: > On 07/24/2018, 05:22 PM, Tetsuo Handa wrote: >> From 118c64e86641a97d44dec39e313a95b12d9bc3b2 Mon Sep 17 00:00:00 2001 >> From: Tetsuo Handa >> Date: Wed, 25 Jul 2018 00:15:18 +0900 >> Subject: [PATCH v2] n_tty: Protect tt

Re: [PATCH v2] n_tty: Protect tty->disc_data using refcount.

2018-08-29 Thread Tetsuo Handa
On 2018/08/29 22:53, Jiri Slaby wrote: > On 07/24/2018, 05:22 PM, Tetsuo Handa wrote: >> From 118c64e86641a97d44dec39e313a95b12d9bc3b2 Mon Sep 17 00:00:00 2001 >> From: Tetsuo Handa >> Date: Wed, 25 Jul 2018 00:15:18 +0900 >> Subject: [PATCH v2] n_tty: Protect tt

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-08-23 Thread Tetsuo Handa
David Rientjes wrote: > On Fri, 24 Aug 2018, Tetsuo Handa wrote: > > > > For those of us who are tracking CVE-2016-10723 which has peristently > > > been > > > labeled as "disputed" and with no clear indication of what patches > > > addre

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-08-23 Thread Tetsuo Handa
David Rientjes wrote: > On Fri, 24 Aug 2018, Tetsuo Handa wrote: > > > > For those of us who are tracking CVE-2016-10723 which has peristently > > > been > > > labeled as "disputed" and with no clear indication of what patches > > > addre

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-08-23 Thread Tetsuo Handa
On 2018/08/24 5:06, David Rientjes wrote: > For those of us who are tracking CVE-2016-10723 which has peristently been > labeled as "disputed" and with no clear indication of what patches address > it, I am assuming that commit 9bfe5ded054b ("mm, oom: remove sleep from > under oom_lock") and

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-08-23 Thread Tetsuo Handa
On 2018/08/24 5:06, David Rientjes wrote: > For those of us who are tracking CVE-2016-10723 which has peristently been > labeled as "disputed" and with no clear indication of what patches address > it, I am assuming that commit 9bfe5ded054b ("mm, oom: remove sleep from > under oom_lock") and

Re: [PATCH] apparmor: remove unused label

2018-08-23 Thread Tetsuo Handa
On 2018/08/23 23:21, Kees Cook wrote: > On Thu, Aug 23, 2018 at 7:09 AM, Arnd Bergmann wrote: >> After the corresponding 'goto' was removed, we get a warning >> for the 'fail' label: >> >> security/apparmor/policy_unpack.c: In function 'unpack_dfa': >> security/apparmor/policy_unpack.c:426:1:

Re: [PATCH] apparmor: remove unused label

2018-08-23 Thread Tetsuo Handa
On 2018/08/23 23:21, Kees Cook wrote: > On Thu, Aug 23, 2018 at 7:09 AM, Arnd Bergmann wrote: >> After the corresponding 'goto' was removed, we get a warning >> for the 'fail' label: >> >> security/apparmor/policy_unpack.c: In function 'unpack_dfa': >> security/apparmor/policy_unpack.c:426:1:

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-08-21 Thread Tetsuo Handa
On 2018/08/03 15:16, Michal Hocko wrote: > On Fri 03-08-18 07:05:54, Tetsuo Handa wrote: >> On 2018/07/31 14:09, Michal Hocko wrote: >>> On Tue 31-07-18 06:01:48, Tetsuo Handa wrote: >>>> On 2018/07/31 4:10, Michal Hocko wrote: >>>>> Since should_r

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-08-21 Thread Tetsuo Handa
On 2018/08/03 15:16, Michal Hocko wrote: > On Fri 03-08-18 07:05:54, Tetsuo Handa wrote: >> On 2018/07/31 14:09, Michal Hocko wrote: >>> On Tue 31-07-18 06:01:48, Tetsuo Handa wrote: >>>> On 2018/07/31 4:10, Michal Hocko wrote: >>>>> Since should_r

Re: INFO: task hung in generic_file_write_iter

2018-08-20 Thread Tetsuo Handa
On 2018/08/06 20:56, Tetsuo Handa wrote: > On 2018/08/06 19:09, Jan Kara wrote: >> Looks like some kind of a race where device block size gets changed while >> getblk() runs (and creates buffers for underlying page). I don't have time >> to nail it down at this moment can have

Re: INFO: task hung in generic_file_write_iter

2018-08-20 Thread Tetsuo Handa
On 2018/08/06 20:56, Tetsuo Handa wrote: > On 2018/08/06 19:09, Jan Kara wrote: >> Looks like some kind of a race where device block size gets changed while >> getblk() runs (and creates buffers for underlying page). I don't have time >> to nail it down at this moment can have

Re: WARNING in try_charge

2018-08-09 Thread Tetsuo Handa
On 2018/08/10 0:07, Michal Hocko wrote: > On Thu 09-08-18 22:57:43, Tetsuo Handa wrote: >> >From b1f38168f14397c7af9c122cd8207663d96e02ec Mon Sep 17 00:00:00 2001 >> From: Tetsuo Handa >> Date: Thu, 9 Aug 2018 22:49:40 +0900 >> Subject: [PATCH] mm, oom: task_will_

Re: WARNING in try_charge

2018-08-09 Thread Tetsuo Handa
On 2018/08/10 0:07, Michal Hocko wrote: > On Thu 09-08-18 22:57:43, Tetsuo Handa wrote: >> >From b1f38168f14397c7af9c122cd8207663d96e02ec Mon Sep 17 00:00:00 2001 >> From: Tetsuo Handa >> Date: Thu, 9 Aug 2018 22:49:40 +0900 >> Subject: [PATCH] mm, oom: task_will_

Re: WARNING in try_charge

2018-08-09 Thread Tetsuo Handa
>From b1f38168f14397c7af9c122cd8207663d96e02ec Mon Sep 17 00:00:00 2001 From: Tetsuo Handa Date: Thu, 9 Aug 2018 22:49:40 +0900 Subject: [PATCH] mm, oom: task_will_free_mem(current) should retry until memory reserve fails Commit 696453e66630ad45 ("mm, oom: task_will_free_mem sho

Re: WARNING in try_charge

2018-08-09 Thread Tetsuo Handa
>From b1f38168f14397c7af9c122cd8207663d96e02ec Mon Sep 17 00:00:00 2001 From: Tetsuo Handa Date: Thu, 9 Aug 2018 22:49:40 +0900 Subject: [PATCH] mm, oom: task_will_free_mem(current) should retry until memory reserve fails Commit 696453e66630ad45 ("mm, oom: task_will_free_mem sho

Re: [PATCH RFC v2 02/10] mm: Make shrink_slab() lockless

2018-08-09 Thread Tetsuo Handa
On 2018/08/09 18:21, Kirill Tkhai wrote: > 2)SRCU. Pros are there are no the above problems; we will have completely > unlocked and > scalable shrink_slab(). We will also have a possibility to avoid > unregistering delays, > like I did for superblock shrinker. There will be full scalability.

Re: [PATCH RFC v2 02/10] mm: Make shrink_slab() lockless

2018-08-09 Thread Tetsuo Handa
On 2018/08/09 18:21, Kirill Tkhai wrote: > 2)SRCU. Pros are there are no the above problems; we will have completely > unlocked and > scalable shrink_slab(). We will also have a possibility to avoid > unregistering delays, > like I did for superblock shrinker. There will be full scalability.

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-08 Thread Tetsuo Handa
On 2018/08/08 5:38, Tetsuo Handa wrote: > On 2018/08/08 5:19, Johannes Weiner wrote: >> On Tue, Aug 07, 2018 at 07:15:11PM +0900, Tetsuo Handa wrote: >>> On 2018/08/07 16:25, Michal Hocko wrote: >>>> @@ -1703,7 +1703,8 @@ static enum oom_status mem_cgroup_oom(struct

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-08 Thread Tetsuo Handa
On 2018/08/08 5:38, Tetsuo Handa wrote: > On 2018/08/08 5:19, Johannes Weiner wrote: >> On Tue, Aug 07, 2018 at 07:15:11PM +0900, Tetsuo Handa wrote: >>> On 2018/08/07 16:25, Michal Hocko wrote: >>>> @@ -1703,7 +1703,8 @@ static enum oom_status mem_cgroup_oom(struct

Re: [PATCH RFC 02/10] mm: Make shrink_slab() lockless

2018-08-08 Thread Tetsuo Handa
On 2018/08/08 20:51, Kirill Tkhai wrote: > @@ -192,7 +193,6 @@ static int prealloc_memcg_shrinker(struct shrinker > *shrinker) > int id, ret = -ENOMEM; > > down_write(_rwsem); > - /* This may call shrinker, so it must use down_read_trylock() */ > id = idr_alloc(_idr,

Re: [PATCH RFC 02/10] mm: Make shrink_slab() lockless

2018-08-08 Thread Tetsuo Handa
On 2018/08/08 20:51, Kirill Tkhai wrote: > @@ -192,7 +193,6 @@ static int prealloc_memcg_shrinker(struct shrinker > *shrinker) > int id, ret = -ENOMEM; > > down_write(_rwsem); > - /* This may call shrinker, so it must use down_read_trylock() */ > id = idr_alloc(_idr,

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-07 Thread Tetsuo Handa
On 2018/08/08 5:19, Johannes Weiner wrote: > On Tue, Aug 07, 2018 at 07:15:11PM +0900, Tetsuo Handa wrote: >> On 2018/08/07 16:25, Michal Hocko wrote: >>> @@ -1703,7 +1703,8 @@ static enum oom_status mem_cgroup_oom(struct >>> mem_cgroup *memcg, gfp_t mask, int >&

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-07 Thread Tetsuo Handa
On 2018/08/08 5:19, Johannes Weiner wrote: > On Tue, Aug 07, 2018 at 07:15:11PM +0900, Tetsuo Handa wrote: >> On 2018/08/07 16:25, Michal Hocko wrote: >>> @@ -1703,7 +1703,8 @@ static enum oom_status mem_cgroup_oom(struct >>> mem_cgroup *memcg, gfp_t mask, int >&

Re: WARNING in try_charge

2018-08-07 Thread Tetsuo Handa
On 2018/08/07 6:50, Tetsuo Handa wrote: > list_for_each_entry_safe(p, tmp, _victim_list, oom_victim_list) { > struct mm_struct *mm = p->signal->oom_mm; > > if (oom_unkillable_task(p, oc->memcg, oc->nodemask)) > co

Re: WARNING in try_charge

2018-08-07 Thread Tetsuo Handa
On 2018/08/07 6:50, Tetsuo Handa wrote: > list_for_each_entry_safe(p, tmp, _victim_list, oom_victim_list) { > struct mm_struct *mm = p->signal->oom_mm; > > if (oom_unkillable_task(p, oc->memcg, oc->nodemask)) > co

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-07 Thread Tetsuo Handa
On 2018/08/07 16:25, Michal Hocko wrote: > @@ -1703,7 +1703,8 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup > *memcg, gfp_t mask, int > return OOM_ASYNC; > } > > - if (mem_cgroup_out_of_memory(memcg, mask, order)) > + if (mem_cgroup_out_of_memory(memcg,

Re: [PATCH] memcg, oom: be careful about races when warning about no reclaimable task

2018-08-07 Thread Tetsuo Handa
On 2018/08/07 16:25, Michal Hocko wrote: > @@ -1703,7 +1703,8 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup > *memcg, gfp_t mask, int > return OOM_ASYNC; > } > > - if (mem_cgroup_out_of_memory(memcg, mask, order)) > + if (mem_cgroup_out_of_memory(memcg,

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/07 5:55, Michal Hocko wrote: > On Tue 07-08-18 05:46:04, Tetsuo Handa wrote: >> On 2018/08/07 5:34, Michal Hocko wrote: >>> On Tue 07-08-18 05:26:23, Tetsuo Handa wrote: >>>> On 2018/08/07 2:56, Michal Hocko wrote: >>>>> So the oom victim ind

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/07 5:55, Michal Hocko wrote: > On Tue 07-08-18 05:46:04, Tetsuo Handa wrote: >> On 2018/08/07 5:34, Michal Hocko wrote: >>> On Tue 07-08-18 05:26:23, Tetsuo Handa wrote: >>>> On 2018/08/07 2:56, Michal Hocko wrote: >>>>> So the oom victim ind

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/07 5:34, Michal Hocko wrote: > On Tue 07-08-18 05:26:23, Tetsuo Handa wrote: >> On 2018/08/07 2:56, Michal Hocko wrote: >>> So the oom victim indeed passed the above force path after the oom >>> invocation. But later on hit the page fault path and t

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/07 5:34, Michal Hocko wrote: > On Tue 07-08-18 05:26:23, Tetsuo Handa wrote: >> On 2018/08/07 2:56, Michal Hocko wrote: >>> So the oom victim indeed passed the above force path after the oom >>> invocation. But later on hit the page fault path and t

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/07 2:56, Michal Hocko wrote: > So the oom victim indeed passed the above force path after the oom > invocation. But later on hit the page fault path and that behaved > differently and for some reason the force path hasn't triggered. I am > wondering how could we hit the page fault path

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/07 2:56, Michal Hocko wrote: > So the oom victim indeed passed the above force path after the oom > invocation. But later on hit the page fault path and that behaved > differently and for some reason the force path hasn't triggered. I am > wondering how could we hit the page fault path

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/07 0:42, syzbot wrote: > Hello, > > syzbot has tested the proposed patch but the reproducer still triggered crash: > WARNING in try_charge > > Killed process 6410 (syz-executor5) total-vm:37708kB, anon-rss:2128kB, > file-rss:0kB, shmem-rss:0kB > oom_reaper: reaped process 6410

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/07 0:42, syzbot wrote: > Hello, > > syzbot has tested the proposed patch but the reproducer still triggered crash: > WARNING in try_charge > > Killed process 6410 (syz-executor5) total-vm:37708kB, anon-rss:2128kB, > file-rss:0kB, shmem-rss:0kB > oom_reaper: reaped process 6410

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
Now, let's test next-20180803 . #syz test: git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 116b181bb646afedd770985de20a68721bdb2648 diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4603ad75c9a9..852cd3dbdcd9 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1388,6

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
Now, let's test next-20180803 . #syz test: git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 116b181bb646afedd770985de20a68721bdb2648 diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4603ad75c9a9..852cd3dbdcd9 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1388,6

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/06 23:58, Michal Hocko wrote: >> Since I can't find mm related changes between next-20180803 (syzbot can >> reproduce) and >> next-20180806 (syzbot has not reproduced), I can't guess what makes this >> problem go away. > > Hmm, but original report was against

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/06 23:58, Michal Hocko wrote: >> Since I can't find mm related changes between next-20180803 (syzbot can >> reproduce) and >> next-20180806 (syzbot has not reproduced), I can't guess what makes this >> problem go away. > > Hmm, but original report was against

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/06 23:54, David Howells wrote: > Do you have a link to the problem? > > David > https://groups.google.com/forum/#!msg/syzkaller-bugs/R03vI7RCVco/0PijCTrcCgAJ syzbot found a reproducer, and the reproducer was working until next-20180803. But the reproducer is failing to reproduce

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/06 23:54, David Howells wrote: > Do you have a link to the problem? > > David > https://groups.google.com/forum/#!msg/syzkaller-bugs/R03vI7RCVco/0PijCTrcCgAJ syzbot found a reproducer, and the reproducer was working until next-20180803. But the reproducer is failing to reproduce

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
+David Howells On 2018/08/06 20:32, Michal Hocko wrote: > On Mon 06-08-18 04:27:02, syzbot wrote: >> Hello, >> >> syzbot has tested the proposed patch and the reproducer did not trigger >> crash: >> >> Reported-and-tested-by: >> syzbot+bab151e82a4e973fa...@syzkaller.appspotmail.com >> >> Tested

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
+David Howells On 2018/08/06 20:32, Michal Hocko wrote: > On Mon 06-08-18 04:27:02, syzbot wrote: >> Hello, >> >> syzbot has tested the proposed patch and the reproducer did not trigger >> crash: >> >> Reported-and-tested-by: >> syzbot+bab151e82a4e973fa...@syzkaller.appspotmail.com >> >> Tested

Re: INFO: task hung in generic_file_write_iter

2018-08-06 Thread Tetsuo Handa
On 2018/08/06 19:09, Jan Kara wrote: >> syzbot reproduced this problem ( >> https://syzkaller.appspot.com/text?tag=CrashLog=11f2fc4440 ) . >> It says that grow_dev_page() is returning 1 but __find_get_block() is >> failing forever. Any idea? So far syzbot reproduced 7 times, and all reports

Re: INFO: task hung in generic_file_write_iter

2018-08-06 Thread Tetsuo Handa
On 2018/08/06 19:09, Jan Kara wrote: >> syzbot reproduced this problem ( >> https://syzkaller.appspot.com/text?tag=CrashLog=11f2fc4440 ) . >> It says that grow_dev_page() is returning 1 but __find_get_block() is >> failing forever. Any idea? So far syzbot reproduced 7 times, and all reports

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/06 19:39, Dmitry Vyukov wrote: > On Mon, Aug 6, 2018 at 11:48 AM, Michal Hocko wrote: >> Btw. running with the above diff on top might help us to ideantify >> whether this is a pre-mature warning or a valid one. Still useful to >> find out. Since syzbot already found a syz reproducer,

Re: WARNING in try_charge

2018-08-06 Thread Tetsuo Handa
On 2018/08/06 19:39, Dmitry Vyukov wrote: > On Mon, Aug 6, 2018 at 11:48 AM, Michal Hocko wrote: >> Btw. running with the above diff on top might help us to ideantify >> whether this is a pre-mature warning or a valid one. Still useful to >> find out. Since syzbot already found a syz reproducer,

Re: WARNING in try_charge

2018-08-05 Thread Tetsuo Handa
On 2018/08/04 22:45, Tetsuo Handa wrote: > syzbot is hitting WARN(1) because of mem_cgroup_out_of_memory() == false. Since syzbot found a syz reproducer, I asked syzbot to try two patches. Setting MMF_OOM_SKIP under oom_lock to prevent from races ( https://syzkaller.appspot.com/x/patch.dif

Re: WARNING in try_charge

2018-08-05 Thread Tetsuo Handa
On 2018/08/04 22:45, Tetsuo Handa wrote: > syzbot is hitting WARN(1) because of mem_cgroup_out_of_memory() == false. Since syzbot found a syz reproducer, I asked syzbot to try two patches. Setting MMF_OOM_SKIP under oom_lock to prevent from races ( https://syzkaller.appspot.com/x/patch.dif

Re: WARNING in try_charge

2018-08-04 Thread Tetsuo Handa
syzbot is hitting WARN(1) because of mem_cgroup_out_of_memory() == false. At first I suspected that syzbot is hitting static bool oom_kill_memcg_victim(struct oom_control *oc) { if (oc->chosen_memcg == NULL || oc->chosen_memcg == INFLIGHT_VICTIM) return

Re: WARNING in try_charge

2018-08-04 Thread Tetsuo Handa
syzbot is hitting WARN(1) because of mem_cgroup_out_of_memory() == false. At first I suspected that syzbot is hitting static bool oom_kill_memcg_victim(struct oom_control *oc) { if (oc->chosen_memcg == NULL || oc->chosen_memcg == INFLIGHT_VICTIM) return

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-08-02 Thread Tetsuo Handa
On 2018/07/31 14:09, Michal Hocko wrote: > On Tue 31-07-18 06:01:48, Tetsuo Handa wrote: >> On 2018/07/31 4:10, Michal Hocko wrote: >>> Since should_reclaim_retry() should be a natural reschedule point, >>> let's do the short sleep for PF_WQ_WORKER threads unconditionall

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-08-02 Thread Tetsuo Handa
On 2018/07/31 14:09, Michal Hocko wrote: > On Tue 31-07-18 06:01:48, Tetsuo Handa wrote: >> On 2018/07/31 4:10, Michal Hocko wrote: >>> Since should_reclaim_retry() should be a natural reschedule point, >>> let's do the short sleep for PF_WQ_WORKER threads unconditionall

Re: [PATCH v2 3/3] mm, oom: introduce memory.oom.group

2018-08-02 Thread Tetsuo Handa
On 2018/08/02 20:21, Michal Hocko wrote: > On Thu 02-08-18 19:53:13, Tetsuo Handa wrote: >> On 2018/08/02 9:32, Roman Gushchin wrote: > [...] >>> +struct mem_cgroup *mem_cgroup_get_oom_group(struct task_struct *victim, >>> +

Re: [PATCH v2 3/3] mm, oom: introduce memory.oom.group

2018-08-02 Thread Tetsuo Handa
On 2018/08/02 20:21, Michal Hocko wrote: > On Thu 02-08-18 19:53:13, Tetsuo Handa wrote: >> On 2018/08/02 9:32, Roman Gushchin wrote: > [...] >>> +struct mem_cgroup *mem_cgroup_get_oom_group(struct task_struct *victim, >>> +

Re: [PATCH v2 3/3] mm, oom: introduce memory.oom.group

2018-08-02 Thread Tetsuo Handa
On 2018/08/02 9:32, Roman Gushchin wrote: > For some workloads an intervention from the OOM killer > can be painful. Killing a random task can bring > the workload into an inconsistent state. > > Historically, there are two common solutions for this > problem: > 1) enabling panic_on_oom, > 2)

Re: [PATCH v2 3/3] mm, oom: introduce memory.oom.group

2018-08-02 Thread Tetsuo Handa
On 2018/08/02 9:32, Roman Gushchin wrote: > For some workloads an intervention from the OOM killer > can be painful. Killing a random task can bring > the workload into an inconsistent state. > > Historically, there are two common solutions for this > problem: > 1) enabling panic_on_oom, > 2)

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-07-31 Thread Tetsuo Handa
On 2018/07/31 20:15, Michal Hocko wrote: >>> I will send the patch to Andrew if the patch is ok. >> >> Andrew, can we send the "we used to have a sleeping point in the oom path >> but this has >> been removed recently" patch to linux.git ? > > This can really wait for the next merge window

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-07-31 Thread Tetsuo Handa
On 2018/07/31 20:15, Michal Hocko wrote: >>> I will send the patch to Andrew if the patch is ok. >> >> Andrew, can we send the "we used to have a sleeping point in the oom path >> but this has >> been removed recently" patch to linux.git ? > > This can really wait for the next merge window

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-07-31 Thread Tetsuo Handa
On 2018/07/31 14:09, Michal Hocko wrote: > On Tue 31-07-18 06:01:48, Tetsuo Handa wrote: >> On 2018/07/31 4:10, Michal Hocko wrote: >>> Since should_reclaim_retry() should be a natural reschedule point, >>> let's do the short sleep for PF_WQ_WORKER threads unconditionall

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-07-31 Thread Tetsuo Handa
On 2018/07/31 14:09, Michal Hocko wrote: > On Tue 31-07-18 06:01:48, Tetsuo Handa wrote: >> On 2018/07/31 4:10, Michal Hocko wrote: >>> Since should_reclaim_retry() should be a natural reschedule point, >>> let's do the short sleep for PF_WQ_WORKER threads unconditionall

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-07-30 Thread Tetsuo Handa
On 2018/07/31 4:10, Michal Hocko wrote: > Since should_reclaim_retry() should be a natural reschedule point, > let's do the short sleep for PF_WQ_WORKER threads unconditionally in > order to guarantee that other pending work items are started. This will > workaround this problem and it is less

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-07-30 Thread Tetsuo Handa
On 2018/07/31 4:10, Michal Hocko wrote: > Since should_reclaim_retry() should be a natural reschedule point, > let's do the short sleep for PF_WQ_WORKER threads unconditionally in > order to guarantee that other pending work items are started. This will > workaround this problem and it is less

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-07-30 Thread Tetsuo Handa
On 2018/07/30 23:54, Tejun Heo wrote: > Hello, > > On Mon, Jul 30, 2018 at 04:46:47PM +0200, Michal Hocko wrote: >> On Mon 30-07-18 23:34:23, Tetsuo Handa wrote: >>> On 2018/07/30 18:32, Michal Hocko wrote: >> [...] >>>> This one is waiting for drai

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-07-30 Thread Tetsuo Handa
On 2018/07/30 23:54, Tejun Heo wrote: > Hello, > > On Mon, Jul 30, 2018 at 04:46:47PM +0200, Michal Hocko wrote: >> On Mon 30-07-18 23:34:23, Tetsuo Handa wrote: >>> On 2018/07/30 18:32, Michal Hocko wrote: >> [...] >>>> This one is waiting for drai

Re: INFO: task hung in generic_file_write_iter

2018-07-30 Thread Tetsuo Handa
On 2018/07/21 5:06, Andrew Morton wrote: > On Fri, 20 Jul 2018 19:36:23 +0900 Tetsuo Handa > wrote: > >>> >>> This report is stalling after mount() completed and process used >>> remap_file_pages(). >>> I think that we might need to use debug pr

Re: INFO: task hung in generic_file_write_iter

2018-07-30 Thread Tetsuo Handa
On 2018/07/21 5:06, Andrew Morton wrote: > On Fri, 20 Jul 2018 19:36:23 +0900 Tetsuo Handa > wrote: > >>> >>> This report is stalling after mount() completed and process used >>> remap_file_pages(). >>> I think that we might need to use debug pr

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-07-30 Thread Tetsuo Handa
32 Mon Sep 17 00:00:00 2001 >>>> From: Michal Hocko >>>> Date: Thu, 26 Jul 2018 14:40:03 +0900 >>>> Subject: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at >>>> should_reclaim_retry(). >>>> >>>> Tetsuo Handa has reported

Re: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at should_reclaim_retry().

2018-07-30 Thread Tetsuo Handa
32 Mon Sep 17 00:00:00 2001 >>>> From: Michal Hocko >>>> Date: Thu, 26 Jul 2018 14:40:03 +0900 >>>> Subject: [PATCH] mm,page_alloc: PF_WQ_WORKER threads must sleep at >>>> should_reclaim_retry(). >>>> >>>> Tetsuo Handa has reported

<    4   5   6   7   8   9   10   11   12   13   >