Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-20 Thread Michal Hocko
On Fri 16-08-19 11:31:45, Jason Gunthorpe wrote: > On Fri, Aug 16, 2019 at 02:26:25PM +0200, Michal Hocko wrote: [...] > > I believe I have given some examples when introducing __GFP_NOLOCKDEP. > > Okay, I think that is 7e7844226f10 ("lockdep: allow to disable reclaim > lockup detection") Hmm,

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Daniel Vetter
On Fri, Aug 16, 2019 at 4:38 PM Jason Gunthorpe wrote: > > On Fri, Aug 16, 2019 at 04:11:34PM +0200, Daniel Vetter wrote: > > Also, aside from this patch (which is prep for the next) and some > > simple reordering conflicts they're all independent. So if there's no > > way to paint this bikeshed

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Jerome Glisse
On Fri, Aug 16, 2019 at 11:31:45AM -0300, Jason Gunthorpe wrote: > On Fri, Aug 16, 2019 at 02:26:25PM +0200, Michal Hocko wrote: > > On Fri 16-08-19 09:19:06, Jason Gunthorpe wrote: > > > On Fri, Aug 16, 2019 at 10:10:29AM +0200, Michal Hocko wrote: > > > > On Thu 15-08-19 17:13:23, Jason

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Daniel Vetter
On Fri, Aug 16, 2019 at 2:12 PM Jason Gunthorpe wrote: > > On Fri, Aug 16, 2019 at 08:20:55AM +0200, Daniel Vetter wrote: > > On Fri, Aug 16, 2019 at 3:00 AM Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 10:49:31PM +0200, Daniel Vetter wrote: > > > > On Thu, Aug 15, 2019 at 10:27 PM Jason

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Michal Hocko
On Fri 16-08-19 09:19:06, Jason Gunthorpe wrote: > On Fri, Aug 16, 2019 at 10:10:29AM +0200, Michal Hocko wrote: > > On Thu 15-08-19 17:13:23, Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 09:35:26PM +0200, Michal Hocko wrote: > > > > > > > > The last detail is I'm still unclear what a GFP

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Michal Hocko
On Thu 15-08-19 22:16:43, Daniel Vetter wrote: > On Thu, Aug 15, 2019 at 9:35 PM Michal Hocko wrote: [...] > > > The last detail is I'm still unclear what a GFP flags a blockable > > > invalidate_range_start() should use. Is GFP_KERNEL OK? > > > > I hope I will not make this muddy again ;) > >

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Michal Hocko
On Thu 15-08-19 15:15:09, Andrew Morton wrote: > On Thu, 15 Aug 2019 10:44:29 +0200 Michal Hocko wrote: > > > > I continue to struggle with this. It introduces a new kernel state > > > "running preemptibly but must not call schedule()". How does this make > > > any sense? > > > > > > Perhaps

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Michal Hocko
On Thu 15-08-19 17:13:23, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 09:35:26PM +0200, Michal Hocko wrote: > > > > The last detail is I'm still unclear what a GFP flags a blockable > > > invalidate_range_start() should use. Is GFP_KERNEL OK? > > > > I hope I will not make this muddy again

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Daniel Vetter
On Fri, Aug 16, 2019 at 3:00 AM Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 10:49:31PM +0200, Daniel Vetter wrote: > > On Thu, Aug 15, 2019 at 10:27 PM Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 10:16:43PM +0200, Daniel Vetter wrote: > > > > So if someone can explain to me how

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Andrew Morton
On Thu, 15 Aug 2019 10:44:29 +0200 Michal Hocko wrote: > > I continue to struggle with this. It introduces a new kernel state > > "running preemptibly but must not call schedule()". How does this make > > any sense? > > > > Perhaps a much, much more detailed description of the oom_reaper > >

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Daniel Vetter
On Thu, Aug 15, 2019 at 10:27 PM Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 10:16:43PM +0200, Daniel Vetter wrote: > > So if someone can explain to me how that works with lockdep I can of > > course implement it. But afaics that doesn't exist (I tried to explain > > that somewhere else

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Daniel Vetter
On Thu, Aug 15, 2019 at 9:35 PM Michal Hocko wrote: > > On Thu 15-08-19 16:18:10, Jason Gunthorpe wrote: > > On Thu, Aug 15, 2019 at 09:05:25PM +0200, Michal Hocko wrote: > > > > > This is what you claim and I am saying that fs_reclaim is about a > > > restricted reclaim context and it is an ugly

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 16:18:10, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 09:05:25PM +0200, Michal Hocko wrote: > > > This is what you claim and I am saying that fs_reclaim is about a > > restricted reclaim context and it is an ugly hack. It has proven to > > report false positives. Maybe it can

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 15:24:48, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 07:42:07PM +0200, Michal Hocko wrote: > > On Thu 15-08-19 13:56:31, Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 06:00:41PM +0200, Michal Hocko wrote: > > > > > > > > AFAIK 'GFP_NOWAIT' is characterized by the lack

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Jerome Glisse
On Thu, Aug 15, 2019 at 03:01:59PM -0300, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 01:39:22PM -0400, Jerome Glisse wrote: > > On Thu, Aug 15, 2019 at 02:35:57PM -0300, Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 06:25:16PM +0200, Daniel Vetter wrote: > > > > > > > I'm not really

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Jerome Glisse
On Thu, Aug 15, 2019 at 07:42:07PM +0200, Michal Hocko wrote: > On Thu 15-08-19 13:56:31, Jason Gunthorpe wrote: > > On Thu, Aug 15, 2019 at 06:00:41PM +0200, Michal Hocko wrote: > > > > > > AFAIK 'GFP_NOWAIT' is characterized by the lack of __GFP_FS and > > > > __GFP_DIRECT_RECLAIM.. > > > > > >

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 13:56:31, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 06:00:41PM +0200, Michal Hocko wrote: > > > > AFAIK 'GFP_NOWAIT' is characterized by the lack of __GFP_FS and > > > __GFP_DIRECT_RECLAIM.. > > > > > > This matches the existing test in __need_fs_reclaim() - so if you are >

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Jerome Glisse
On Thu, Aug 15, 2019 at 02:35:57PM -0300, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 06:25:16PM +0200, Daniel Vetter wrote: > > > I'm not really well versed in the details of our userptr, but both > > amdgpu and i915 wait for the gpu to complete from > > invalidate_range_start. Jerome has

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Jerome Glisse
On Thu, Aug 15, 2019 at 07:21:47PM +0200, Daniel Vetter wrote: > On Thu, Aug 15, 2019 at 7:16 PM Jason Gunthorpe wrote: > > > > On Thu, Aug 15, 2019 at 12:32:38PM -0400, Jerome Glisse wrote: > > > On Thu, Aug 15, 2019 at 12:10:28PM -0300, Jason Gunthorpe wrote: > > > > On Thu, Aug 15, 2019 at

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Daniel Vetter
On Thu, Aug 15, 2019 at 7:16 PM Jason Gunthorpe wrote: > > On Thu, Aug 15, 2019 at 12:32:38PM -0400, Jerome Glisse wrote: > > On Thu, Aug 15, 2019 at 12:10:28PM -0300, Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 04:43:38PM +0200, Daniel Vetter wrote: > > > > > > > You have to wait for

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Jerome Glisse
On Thu, Aug 15, 2019 at 01:56:31PM -0300, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 06:00:41PM +0200, Michal Hocko wrote: > > > > AFAIK 'GFP_NOWAIT' is characterized by the lack of __GFP_FS and > > > __GFP_DIRECT_RECLAIM.. > > > > > > This matches the existing test in __need_fs_reclaim() -

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Jerome Glisse
On Thu, Aug 15, 2019 at 12:10:28PM -0300, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 04:43:38PM +0200, Daniel Vetter wrote: > > > You have to wait for the gpu to finnish current processing in > > invalidate_range_start. Otherwise there's no point to any of this > > really. So the

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Daniel Vetter
On Thu, Aug 15, 2019 at 5:10 PM Jason Gunthorpe wrote: > > On Thu, Aug 15, 2019 at 04:43:38PM +0200, Daniel Vetter wrote: > > > You have to wait for the gpu to finnish current processing in > > invalidate_range_start. Otherwise there's no point to any of this > > really. So the

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 11:12:19, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 03:21:27PM +0200, Michal Hocko wrote: > > On Thu 15-08-19 09:23:44, Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 08:58:29AM +0200, Daniel Vetter wrote: > > > > On Wed, Aug 14, 2019 at 08:58:05PM -0300, Jason

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Daniel Vetter
On Thu, Aug 15, 2019 at 4:38 PM Jason Gunthorpe wrote: > > On Thu, Aug 15, 2019 at 03:12:11PM +0200, Daniel Vetter wrote: > > On Thu, Aug 15, 2019 at 3:04 PM Jason Gunthorpe wrote: > > > > > > On Thu, Aug 15, 2019 at 10:44:29AM +0200, Michal Hocko wrote: > > > > > > > As the oom reaper is the

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 10:04:15, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 10:44:29AM +0200, Michal Hocko wrote: > > > As the oom reaper is the primary guarantee of the oom handling forward > > progress it cannot be blocked on anything that might depend on blockable > > memory allocations. These

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 09:23:44, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 08:58:29AM +0200, Daniel Vetter wrote: > > On Wed, Aug 14, 2019 at 08:58:05PM -0300, Jason Gunthorpe wrote: > > > On Wed, Aug 14, 2019 at 10:20:24PM +0200, Daniel Vetter wrote: > > > > In some special cases we must not

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Daniel Vetter
On Thu, Aug 15, 2019 at 3:04 PM Jason Gunthorpe wrote: > > On Thu, Aug 15, 2019 at 10:44:29AM +0200, Michal Hocko wrote: > > > As the oom reaper is the primary guarantee of the oom handling forward > > progress it cannot be blocked on anything that might depend on blockable > > memory

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Wed 14-08-19 13:45:58, Andrew Morton wrote: > On Wed, 14 Aug 2019 22:20:24 +0200 Daniel Vetter > wrote: > > > In some special cases we must not block, but there's not a > > spinlock, preempt-off, irqs-off or similar critical section already > > that arms the might_sleep() debug checks. Add a

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Daniel Vetter
On Wed, Aug 14, 2019 at 08:58:05PM -0300, Jason Gunthorpe wrote: > On Wed, Aug 14, 2019 at 10:20:24PM +0200, Daniel Vetter wrote: > > In some special cases we must not block, but there's not a > > spinlock, preempt-off, irqs-off or similar critical section already > > that arms the might_sleep()

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Daniel Vetter
On Wed, Aug 14, 2019 at 01:45:58PM -0700, Andrew Morton wrote: > On Wed, 14 Aug 2019 22:20:24 +0200 Daniel Vetter > wrote: > > > In some special cases we must not block, but there's not a > > spinlock, preempt-off, irqs-off or similar critical section already > > that arms the might_sleep()

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-14 Thread Andrew Morton
On Wed, 14 Aug 2019 22:20:24 +0200 Daniel Vetter wrote: > In some special cases we must not block, but there's not a > spinlock, preempt-off, irqs-off or similar critical section already > that arms the might_sleep() debug checks. Add a non_block_start/end() > pair to annotate these. > > This