Re: [PATCH v3 1/4] mm/mmu_notifier: Allow two-pass struct mmu_interval_notifiers
On Wed, 2026-03-04 at 20:45 +0100, David Hildenbrand (Arm) wrote:
> On 3/3/26 14:34, Thomas Hellström wrote:
> > GPU use-cases for mmu_interval_notifiers with hmm often involve
> > starting a gpu operation and then waiting for it to complete.
> > These operations are typically context preemption or TLB flushing.
> >
> > With single-pass notifiers per GPU this doesn't scale in
> > multi-gpu scenarios. In those scenarios we'd want to first start
> > preemption- or TLB flushing on all GPUs and as a second pass wait
> > for them to complete.
> >
> > One can do this on per-driver basis multiplexing per-driver
> > notifiers but that would mean sharing the notifier "user" lock
> > across all GPUs and that doesn't scale well either, so adding
> > support
> > for multi-pass in the core appears to be the right choice.
> >
> > Implement two-pass capability in the mmu_interval_notifier. Use a
> > linked list for the final passes to minimize the impact for
> > use-cases that don't need the multi-pass functionality by avoiding
> > a second interval tree walk, and to be able to easily pass data
> > between the two passes.
> >
> > v1:
> > - Restrict to two passes (Jason Gunthorpe)
> > - Improve on documentation (Jason Gunthorpe)
> > - Improve on function naming (Alistair Popple)
> > v2:
> > - Include the invalidate_finish() callback in the
> > struct mmu_interval_notifier_ops.
> > - Update documentation (GitHub Copilot:claude-sonnet-4.6)
> > - Use lockless list for list management.
> > v3:
> > - Update kerneldoc for the struct
> > mmu_interval_notifier_finish::list member
> > (Matthew Brost)
> > - Add a WARN_ON_ONCE() checking for NULL invalidate_finish() op if
> > if invalidate_start() is non-NULL. (Matthew Brost)
> >
> > Cc: Jason Gunthorpe
> > Cc: Andrew Morton
> > Cc: Simona Vetter
> > Cc: Dave Airlie
> > Cc: Alistair Popple
> > Cc:
> > Cc:
> > Cc:
> >
> > Assisted-by: GitHub Copilot:claude-sonnet-4.6 # Documentation only.
> > Signed-off-by: Thomas Hellström
> > ---
> > include/linux/mmu_notifier.h | 38 +
> > mm/mmu_notifier.c | 65 +++-
> >
> > 2 files changed, 94 insertions(+), 9 deletions(-)
> >
> > diff --git a/include/linux/mmu_notifier.h
> > b/include/linux/mmu_notifier.h
> > index 07a2bbaf86e9..37b683163235 100644
> > --- a/include/linux/mmu_notifier.h
> > +++ b/include/linux/mmu_notifier.h
> > @@ -233,16 +233,54 @@ struct mmu_notifier {
> > unsigned int users;
> > };
> >
> > +/**
> > + * struct mmu_interval_notifier_finish - mmu_interval_notifier
> > two-pass abstraction
> > + * @link: Lockless list link for the notifiers pending pass list
> > + * @notifier: The mmu_interval_notifier for which the finish pass
> > is called.
> > + *
> > + * Allocate, typically using GFP_NOWAIT in the interval notifier's
> > first pass.
>
> Might want to make it clear that the fist pass is "start" and the
> second
> pass is "finish".
>
> Two-pass makes it sound like we'd be calling the same operation
> (e.g.,
> invalidate() ) twice.
>
> > + * If allocation fails (which is not unlikely under memory
> > pressure), fall back
> > + * to single-pass operation.
>
> Do you mean that the core will fallback (calling invalidate() ) or
> that
> it's the responsibility of the notifier to behave as if invalidate()
> would be called to then return finish=NULL? I assume the latter.
>
> Maybe this should be documented for @invalidate_start instead.
> (behave
> like invalidate() if @finish is %NULL on return etc)
>
> > Note that with a large number of notifiers
> > + * implementing two passes, allocation with GFP_NOWAIT will become
> > increasingly
> > + * likely to fail, so consider implementing a small pool instead
> > of using
> > + * kmalloc() allocations.
> > + *
> > + * If the implementation needs to pass data between the two
> > passes,
> > + * the recommended way is to embed struct
> > mmu_interval_notifier_finish into a larger
> > + * structure that also contains the data needed to be shared. Keep
> > in mind that
> > + * a notifier callback can be invoked in parallel, and each
> > invocation needs its
> > + * own struct mmu_interval_notifier_finish.
> > + */
> > +struct mmu_interval_notifier_finish {
> > + struct llist_node link;
> > + struct mmu_interval_notifier *notifier;
> > +};
> > +
> > /**
> > * struct mmu_interval_notifier_ops
> > * @invalidate: Upon return the caller must stop using any SPTEs
> > within this
> > * range. This function can sleep. Return false only
> > if sleeping
> > * was required but
> > mmu_notifier_range_blockable(range) is false.
> > + * @invalidate_start: Similar to @invalidate, but intended for
> > two-pass notifier
> > + * callbacks where the call to
> > @invalidate_start is the first
> > + * pass and any struct
> > mmu_interval_notifier_finish pointer
> > + * returned in the @finish parameter describes
> > t
Re: [PATCH v3 1/4] mm/mmu_notifier: Allow two-pass struct mmu_interval_notifiers
On Wed, 2026-03-04 at 20:32 +0100, David Hildenbrand (Arm) wrote: > On 3/3/26 14:34, Thomas Hellström wrote: > > GPU use-cases for mmu_interval_notifiers with hmm often involve > > starting a gpu operation and then waiting for it to complete. > > These operations are typically context preemption or TLB flushing. > > > > With single-pass notifiers per GPU this doesn't scale in > > multi-gpu scenarios. In those scenarios we'd want to first start > > preemption- or TLB flushing on all GPUs and as a second pass wait > > for them to complete. > > > > One can do this on per-driver basis multiplexing per-driver > > notifiers but that would mean sharing the notifier "user" lock > > across all GPUs and that doesn't scale well either, so adding > > support > > for multi-pass in the core appears to be the right choice. > > > > Implement two-pass capability in the mmu_interval_notifier. Use a > > linked list for the final passes to minimize the impact for > > use-cases that don't need the multi-pass functionality by avoiding > > a second interval tree walk, and to be able to easily pass data > > between the two passes. > > Please CC all maintainers+reviewers that MAINTAINERS recommends you > to cc. Hmm. Good point. I missed a fair number of those. Will Resend Thanks, Thomas
Re: [PATCH v3 1/4] mm/mmu_notifier: Allow two-pass struct mmu_interval_notifiers
On 3/3/26 14:34, Thomas Hellström wrote:
> GPU use-cases for mmu_interval_notifiers with hmm often involve
> starting a gpu operation and then waiting for it to complete.
> These operations are typically context preemption or TLB flushing.
>
> With single-pass notifiers per GPU this doesn't scale in
> multi-gpu scenarios. In those scenarios we'd want to first start
> preemption- or TLB flushing on all GPUs and as a second pass wait
> for them to complete.
>
> One can do this on per-driver basis multiplexing per-driver
> notifiers but that would mean sharing the notifier "user" lock
> across all GPUs and that doesn't scale well either, so adding support
> for multi-pass in the core appears to be the right choice.
>
> Implement two-pass capability in the mmu_interval_notifier. Use a
> linked list for the final passes to minimize the impact for
> use-cases that don't need the multi-pass functionality by avoiding
> a second interval tree walk, and to be able to easily pass data
> between the two passes.
>
> v1:
> - Restrict to two passes (Jason Gunthorpe)
> - Improve on documentation (Jason Gunthorpe)
> - Improve on function naming (Alistair Popple)
> v2:
> - Include the invalidate_finish() callback in the
> struct mmu_interval_notifier_ops.
> - Update documentation (GitHub Copilot:claude-sonnet-4.6)
> - Use lockless list for list management.
> v3:
> - Update kerneldoc for the struct mmu_interval_notifier_finish::list member
> (Matthew Brost)
> - Add a WARN_ON_ONCE() checking for NULL invalidate_finish() op if
> if invalidate_start() is non-NULL. (Matthew Brost)
>
> Cc: Jason Gunthorpe
> Cc: Andrew Morton
> Cc: Simona Vetter
> Cc: Dave Airlie
> Cc: Alistair Popple
> Cc:
> Cc:
> Cc:
>
> Assisted-by: GitHub Copilot:claude-sonnet-4.6 # Documentation only.
> Signed-off-by: Thomas Hellström
> ---
> include/linux/mmu_notifier.h | 38 +
> mm/mmu_notifier.c| 65 +++-
> 2 files changed, 94 insertions(+), 9 deletions(-)
>
> diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h
> index 07a2bbaf86e9..37b683163235 100644
> --- a/include/linux/mmu_notifier.h
> +++ b/include/linux/mmu_notifier.h
> @@ -233,16 +233,54 @@ struct mmu_notifier {
> unsigned int users;
> };
>
> +/**
> + * struct mmu_interval_notifier_finish - mmu_interval_notifier two-pass
> abstraction
> + * @link: Lockless list link for the notifiers pending pass list
> + * @notifier: The mmu_interval_notifier for which the finish pass is called.
> + *
> + * Allocate, typically using GFP_NOWAIT in the interval notifier's first
> pass.
Might want to make it clear that the fist pass is "start" and the second
pass is "finish".
Two-pass makes it sound like we'd be calling the same operation (e.g.,
invalidate() ) twice.
> + * If allocation fails (which is not unlikely under memory pressure), fall
> back
> + * to single-pass operation.
Do you mean that the core will fallback (calling invalidate() ) or that
it's the responsibility of the notifier to behave as if invalidate()
would be called to then return finish=NULL? I assume the latter.
Maybe this should be documented for @invalidate_start instead. (behave
like invalidate() if @finish is %NULL on return etc)
> Note that with a large number of notifiers
> + * implementing two passes, allocation with GFP_NOWAIT will become
> increasingly
> + * likely to fail, so consider implementing a small pool instead of using
> + * kmalloc() allocations.
> + *
> + * If the implementation needs to pass data between the two passes,
> + * the recommended way is to embed struct mmu_interval_notifier_finish into
> a larger
> + * structure that also contains the data needed to be shared. Keep in mind
> that
> + * a notifier callback can be invoked in parallel, and each invocation needs
> its
> + * own struct mmu_interval_notifier_finish.
> + */
> +struct mmu_interval_notifier_finish {
> + struct llist_node link;
> + struct mmu_interval_notifier *notifier;
> +};
> +
> /**
> * struct mmu_interval_notifier_ops
> * @invalidate: Upon return the caller must stop using any SPTEs within this
> * range. This function can sleep. Return false only if sleeping
> * was required but mmu_notifier_range_blockable(range) is
> false.
> + * @invalidate_start: Similar to @invalidate, but intended for two-pass
> notifier
> + *callbacks where the call to @invalidate_start is the
> first
> + *pass and any struct mmu_interval_notifier_finish
> pointer
> + *returned in the @finish parameter describes the final
> pass.
> + *If @finish is %NULL on return, then no final pass will
> be
> + *called.
Is @finish guaranteed to be set to %NULL before the call? The existing
code does it, but is it something notifiers can rely on?
> + * @invalidate_finish: Called as the second pass for any notifier
Re: [PATCH v3 1/4] mm/mmu_notifier: Allow two-pass struct mmu_interval_notifiers
On 3/3/26 14:34, Thomas Hellström wrote: > GPU use-cases for mmu_interval_notifiers with hmm often involve > starting a gpu operation and then waiting for it to complete. > These operations are typically context preemption or TLB flushing. > > With single-pass notifiers per GPU this doesn't scale in > multi-gpu scenarios. In those scenarios we'd want to first start > preemption- or TLB flushing on all GPUs and as a second pass wait > for them to complete. > > One can do this on per-driver basis multiplexing per-driver > notifiers but that would mean sharing the notifier "user" lock > across all GPUs and that doesn't scale well either, so adding support > for multi-pass in the core appears to be the right choice. > > Implement two-pass capability in the mmu_interval_notifier. Use a > linked list for the final passes to minimize the impact for > use-cases that don't need the multi-pass functionality by avoiding > a second interval tree walk, and to be able to easily pass data > between the two passes. Please CC all maintainers+reviewers that MAINTAINERS recommends you to cc. -- Cheers, David
