Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-09-06 Thread Tetsuo Handa
On 2018/08/27 16:41, Christian König wrote: > Am 26.08.2018 um 10:40 schrieb Tetsuo Handa: >> I'm not following. Why don't we need to do like below (given that >> nobody except amdgpu_mn_read_lock() holds ->read_lock) because e.g. >> drm_sched_fence_create() from drm_sched_job_init() from

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-27 Thread Christian König
Am 26.08.2018 um 10:40 schrieb Tetsuo Handa: On 2018/08/24 22:52, Michal Hocko wrote: @@ -180,11 +180,15 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn) */ static int amdgpu_mn_read_lock(struct amdgpu_mn *amn, bool blockable) { - if (blockable) -

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-26 Thread Tetsuo Handa
On 2018/08/24 22:52, Michal Hocko wrote: > @@ -180,11 +180,15 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn) > */ > static int amdgpu_mn_read_lock(struct amdgpu_mn *amn, bool blockable) > { > - if (blockable) > - mutex_lock(>read_lock); > - else if

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Jerome Glisse
On Fri, Aug 24, 2018 at 06:40:03PM +0200, Michal Hocko wrote: > On Fri 24-08-18 11:12:40, Jerome Glisse wrote: > [...] > > I am fine with Michal patch, i already said so couple month ago first time > > this discussion did pop up, Michal you can add: > > > > Reviewed-by: Jérôme Glisse > > So I

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 11:12:40, Jerome Glisse wrote: [...] > I am fine with Michal patch, i already said so couple month ago first time > this discussion did pop up, Michal you can add: > > Reviewed-by: Jérôme Glisse So I guess the below is the patch you were talking about? From

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Jerome Glisse
On Fri, Aug 24, 2018 at 11:52:25PM +0900, Tetsuo Handa wrote: > On 2018/08/24 22:32, Michal Hocko wrote: > > On Fri 24-08-18 22:02:23, Tetsuo Handa wrote: > >> I worry that (currently > >> out-of-tree) users of this API are involving work / recursion. > > > > I do not give a slightest about

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Jerome Glisse
On Fri, Aug 24, 2018 at 02:33:41PM +0200, Michal Hocko wrote: > On Fri 24-08-18 14:18:44, Christian König wrote: > > Am 24.08.2018 um 14:03 schrieb Michal Hocko: > > > On Fri 24-08-18 13:57:52, Christian König wrote: > > > > Am 24.08.2018 um 13:52 schrieb Michal Hocko: > > > > > On Fri 24-08-18

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Tetsuo Handa
On 2018/08/24 22:32, Michal Hocko wrote: > On Fri 24-08-18 22:02:23, Tetsuo Handa wrote: >> I worry that (currently >> out-of-tree) users of this API are involving work / recursion. > > I do not give a slightest about out-of-tree modules. They will have to > accomodate to the new API. I have no

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Jerome Glisse
On Fri, Aug 24, 2018 at 07:54:19PM +0900, Tetsuo Handa wrote: > Two more worries for this patch. [...] > > > --- a/mm/hmm.c > > +++ b/mm/hmm.c > > @@ -177,16 +177,19 @@ static void hmm_release(struct mmu_notifier *mn, > > struct mm_struct *mm) > > up_write(>mirrors_sem); > > } > > >

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 15:44:03, Christian König wrote: > Am 24.08.2018 um 15:40 schrieb Michal Hocko: > > On Fri 24-08-18 15:28:33, Christian König wrote: > > > Am 24.08.2018 um 15:24 schrieb Michal Hocko: > > > > On Fri 24-08-18 15:10:08, Christian König wrote: > > > > > Am 24.08.2018 um 15:01 schrieb

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Christian König
Am 24.08.2018 um 15:40 schrieb Michal Hocko: On Fri 24-08-18 15:28:33, Christian König wrote: Am 24.08.2018 um 15:24 schrieb Michal Hocko: On Fri 24-08-18 15:10:08, Christian König wrote: Am 24.08.2018 um 15:01 schrieb Michal Hocko: On Fri 24-08-18 14:52:26, Christian König wrote: Am

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 15:28:33, Christian König wrote: > Am 24.08.2018 um 15:24 schrieb Michal Hocko: > > On Fri 24-08-18 15:10:08, Christian König wrote: > > > Am 24.08.2018 um 15:01 schrieb Michal Hocko: > > > > On Fri 24-08-18 14:52:26, Christian König wrote: > > > > > Am 24.08.2018 um 14:33 schrieb

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 22:02:23, Tetsuo Handa wrote: > On 2018/08/24 20:36, Michal Hocko wrote: > >> That is, this API seems to be currently used by only out-of-tree users. > >> Since > >> we can't check that nobody has memory allocation dependency, I think that > >> hmm_invalidate_range_start() should

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Christian König
Am 24.08.2018 um 15:24 schrieb Michal Hocko: On Fri 24-08-18 15:10:08, Christian König wrote: Am 24.08.2018 um 15:01 schrieb Michal Hocko: On Fri 24-08-18 14:52:26, Christian König wrote: Am 24.08.2018 um 14:33 schrieb Michal Hocko: [...] Thiking about it some more, I can imagine that a

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 15:10:08, Christian König wrote: > Am 24.08.2018 um 15:01 schrieb Michal Hocko: > > On Fri 24-08-18 14:52:26, Christian König wrote: > > > Am 24.08.2018 um 14:33 schrieb Michal Hocko: > > [...] > > > > Thiking about it some more, I can imagine that a notifier callback which > > > >

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Christian König
Am 24.08.2018 um 15:01 schrieb Michal Hocko: On Fri 24-08-18 14:52:26, Christian König wrote: Am 24.08.2018 um 14:33 schrieb Michal Hocko: [...] Thiking about it some more, I can imagine that a notifier callback which performs an allocation might trigger a memory reclaim and that in turn

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Tetsuo Handa
On 2018/08/24 20:36, Michal Hocko wrote: >> That is, this API seems to be currently used by only out-of-tree users. Since >> we can't check that nobody has memory allocation dependency, I think that >> hmm_invalidate_range_start() should return -EAGAIN if blockable == false for >> now. > > The

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 14:52:26, Christian König wrote: > Am 24.08.2018 um 14:33 schrieb Michal Hocko: [...] > > Thiking about it some more, I can imagine that a notifier callback which > > performs an allocation might trigger a memory reclaim and that in turn > > might trigger a notifier to be invoked

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Christian König
Am 24.08.2018 um 14:33 schrieb Michal Hocko: On Fri 24-08-18 14:18:44, Christian König wrote: Am 24.08.2018 um 14:03 schrieb Michal Hocko: On Fri 24-08-18 13:57:52, Christian König wrote: Am 24.08.2018 um 13:52 schrieb Michal Hocko: On Fri 24-08-18 13:43:16, Christian König wrote: [...]

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 14:18:44, Christian König wrote: > Am 24.08.2018 um 14:03 schrieb Michal Hocko: > > On Fri 24-08-18 13:57:52, Christian König wrote: > > > Am 24.08.2018 um 13:52 schrieb Michal Hocko: > > > > On Fri 24-08-18 13:43:16, Christian König wrote: > > [...] > > > > > That won't work like

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Christian König
Am 24.08.2018 um 14:03 schrieb Michal Hocko: On Fri 24-08-18 13:57:52, Christian König wrote: Am 24.08.2018 um 13:52 schrieb Michal Hocko: On Fri 24-08-18 13:43:16, Christian König wrote: [...] That won't work like this there might be multiple invalidate_range_start()/invalidate_range_end()

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 13:57:52, Christian König wrote: > Am 24.08.2018 um 13:52 schrieb Michal Hocko: > > On Fri 24-08-18 13:43:16, Christian König wrote: [...] > > > That won't work like this there might be multiple > > > invalidate_range_start()/invalidate_range_end() pairs open at the same > > >

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Christian König
Am 24.08.2018 um 13:52 schrieb Michal Hocko: On Fri 24-08-18 13:43:16, Christian König wrote: Am 24.08.2018 um 13:32 schrieb Michal Hocko: On Fri 24-08-18 19:54:19, Tetsuo Handa wrote: Two more worries for this patch. --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +++

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 13:43:16, Christian König wrote: > Am 24.08.2018 um 13:32 schrieb Michal Hocko: > > On Fri 24-08-18 19:54:19, Tetsuo Handa wrote: > > > Two more worries for this patch. > > > > > > > > > > > > > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c > > > > +++

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Christian König
Am 24.08.2018 um 13:32 schrieb Michal Hocko: On Fri 24-08-18 19:54:19, Tetsuo Handa wrote: Two more worries for this patch. --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c @@ -178,12 +178,18 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn) * *

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 19:54:19, Tetsuo Handa wrote: [...] > > --- a/mm/hmm.c > > +++ b/mm/hmm.c > > @@ -177,16 +177,19 @@ static void hmm_release(struct mmu_notifier *mn, > > struct mm_struct *mm) > > up_write(>mirrors_sem); > > } > > > > -static void hmm_invalidate_range_start(struct

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 19:54:19, Tetsuo Handa wrote: > Two more worries for this patch. > > > > > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c > > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c > > @@ -178,12 +178,18 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn) > > * > > * @amn: our notifier > >

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Tetsuo Handa
Two more worries for this patch. > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c > @@ -178,12 +178,18 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn) > * > * @amn: our notifier > */ > -static void amdgpu_mn_read_lock(struct amdgpu_mn *amn) >

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-25 Thread Michal Hocko
On Tue 24-07-18 12:53:07, Andrew Morton wrote: [...] > > On top of that the proposed cleanup looks as follows: > > > > Looks good to me. Seems a bit strange that we omit the pr_info() > output if the mm was partially reaped - people would still want to know > this? Not very important though.

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-25 Thread Michal Hocko
On Tue 24-07-18 14:07:49, David Rientjes wrote: [...] > mm/oom_kill.c: clean up oom_reap_task_mm() fix > > indicate reaping has been partially skipped so we can expect future skips > or another start before finish. But we are not skipping. This is essentially the same case as mmap_sem trylock

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-24 Thread David Rientjes
On Tue, 24 Jul 2018, Michal Hocko wrote: > oom_reap_task_mm should return false when __oom_reap_task_mm return > false. This is what my patch did but it seems this changed by > http://www.ozlabs.org/~akpm/mmotm/broken-out/mm-oom-remove-oom_lock-from-oom_reaper.patch > so that one should be fixed.

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-24 Thread Andrew Morton
On Tue, 24 Jul 2018 16:17:47 +0200 Michal Hocko wrote: > On Fri 20-07-18 17:09:02, Andrew Morton wrote: > [...] > > - Undocumented return value. > > > > - comment "failed to reap part..." is misleading - sounds like it's > > referring to something which happened in the past, is in fact > >

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-24 Thread Michal Hocko
On Fri 20-07-18 17:09:02, Andrew Morton wrote: [...] > - Undocumented return value. > > - comment "failed to reap part..." is misleading - sounds like it's > referring to something which happened in the past, is in fact > referring to something which might happen in the future. > > - fails

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-23 Thread Michal Hocko
On Fri 20-07-18 16:01:25, Andrew Morton wrote: > On Tue, 17 Jul 2018 10:12:01 +0200 Michal Hocko wrote: > > > > Any suggestions regarding how the driver developers can test this code > > > path? I don't think we presently have a way to fake an oom-killing > > > event? Perhaps we should add

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-23 Thread Michal Hocko
On Mon 23-07-18 09:11:54, Michal Hocko wrote: > On Mon 23-07-18 09:03:06, Michal Hocko wrote: > > On Fri 20-07-18 17:09:02, Andrew Morton wrote: > > [...] > > > Please take a look? > > > > Are you OK to have these in a separate patch? > > Btw. I will rebase this patch once oom stuff in

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-23 Thread Michal Hocko
On Mon 23-07-18 09:03:06, Michal Hocko wrote: > On Fri 20-07-18 17:09:02, Andrew Morton wrote: > [...] > > Please take a look? > > Are you OK to have these in a separate patch? Btw. I will rebase this patch once oom stuff in linux-next settles. At least oom_lock removal from oom_reaper will

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-23 Thread Michal Hocko
On Fri 20-07-18 17:09:02, Andrew Morton wrote: [...] > Please take a look? Are you OK to have these in a separate patch? -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel@lists.xenproject.org

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-20 Thread Andrew Morton
On Mon, 16 Jul 2018 13:50:58 +0200 Michal Hocko wrote: > From: Michal Hocko > > There are several blockable mmu notifiers which might sleep in > mmu_notifier_invalidate_range_start and that is a problem for the > oom_reaper because it needs to guarantee a forward progress so it cannot > depend

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-20 Thread Andrew Morton
On Tue, 17 Jul 2018 10:12:01 +0200 Michal Hocko wrote: > > Any suggestions regarding how the driver developers can test this code > > path? I don't think we presently have a way to fake an oom-killing > > event? Perhaps we should add such a thing, given the problems we're > > having with that

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-17 Thread Michal Hocko
On Mon 16-07-18 16:12:49, Andrew Morton wrote: > On Mon, 16 Jul 2018 13:50:58 +0200 Michal Hocko wrote: > > > From: Michal Hocko > > > > There are several blockable mmu notifiers which might sleep in > > mmu_notifier_invalidate_range_start and that is a problem for the > > oom_reaper because

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-16 Thread Leon Romanovsky
On Mon, Jul 16, 2018 at 04:12:49PM -0700, Andrew Morton wrote: > On Mon, 16 Jul 2018 13:50:58 +0200 Michal Hocko wrote: > > > From: Michal Hocko > > > > There are several blockable mmu notifiers which might sleep in > > mmu_notifier_invalidate_range_start and that is a problem for the > >

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-16 Thread Andrew Morton
On Mon, 16 Jul 2018 13:50:58 +0200 Michal Hocko wrote: > From: Michal Hocko > > There are several blockable mmu notifiers which might sleep in > mmu_notifier_invalidate_range_start and that is a problem for the > oom_reaper because it needs to guarantee a forward progress so it cannot > depend

[Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-16 Thread Michal Hocko
From: Michal Hocko There are several blockable mmu notifiers which might sleep in mmu_notifier_invalidate_range_start and that is a problem for the oom_reaper because it needs to guarantee a forward progress so it cannot depend on any sleepable locks. Currently we simply back off and mark an