On Wed, Jul 08, 2026 at 09:00:03PM +0200, Sebastian Andrzej Siewior wrote:
> -*, +module +Paul.
>
> On 2026-07-08 15:56:53 [+0200], To Petr Pavlu wrote:
> > > One problem is that I'm not sure where the new rcu_barrier() call should
> > > be placed. The prototype adds it before calling the module's exit
> > > function. Would this actually fit all modules? From a quick look, I can
> > > see that various modules call it at different points during their exit.
> >
> > I don't know why you would use call_rcu() in your module_exit()
> > (pointing to the same module). But you could have call_rcu() invoking
> > kmem_cache_free() and destroying that cache (kmem_cache_destroy()) in
> > your exit path. From that perspective it would make sense to flush all
> > calls before invoking module_exit().
>
> Paul, are the RCU callbacks always invoked in FIFO order?
> If we put the module unmap into a call_rcu() (instead of the current
> synchronize_rcu()) would we invoke the callback's of the module's
> callback before the unmap of the module? Or is this not guaranteed due
> callbacks on CPU0 vs CPU1 are executed in different order?
The RCU-callback ordering guarantees are quite weak:
o If callback A executes call_rcu() that queues callback B, then
A will be invoked before B. (Just in case anyone had any doubt.)
o The callback queued prior to a call to rcu_barrier() will be
invoked before any that are queued after return from that same
call to rcu_barrier().
The current *implementation* orders callbacks queued on a given CPU
(but please see below), but there are absolutely no ordering guarantees
among CPUs. To see why, consider the following sequence of events:
o CPU 0 does an "rm -rf" of a large file tree containing huge
numbers of small files. This results in an RCU callback being
queued for each file.
o CPU 0 queues RCU callback A.
o The corresponding grace period completes, so CPU 0 starts
invoking callbacks.
o CPU 1 queues RCU callback B.
o The corresponding grace period completes, so CPU 1 invokes
callback B.
o Meanwhile, CPU 0 is still working off its RCU callback backlog.
o CPU 0 eventually invokes RCU callback A
Worse yet, if we here in RCU-land ever allow a given CPU's callbacks
to be offloaded or deoffloaded while that CPU is online, then there
really will be cases where callbacks queued even by a single given CPU
get invoked out of order. And there are people who would dearly love
us to make this happen. (The last time we tried, we were inundated in
odd race conditions, but maybe we will come up with a better way.)
So these RCU callback ordering guarantees are probably too weak for your
use case.
> Because if the FIFO order is guaranteed then it would be cheapest
> solution. But it would require a rcu_barrier() in kmem_cache_destroy()).
What you maybe *could* do is to have the two RCU callbacks communicate,
so that the last one to be invoked did the work of both of them. For
example, use a shared variable initialized to 2, then have each callback
do atomic_dec_and_test(), with the "winner" doing the work.
Would that do the trick?
Thanx, Paul
