On 8/27/26 08:53, Zqiang wrote:
Based on your description below, the 5.srcu_barrier() did not intercept the
callback of 4.call_srcu(),
this means that 4.call_srcu() and 5.srcu_barrier() concurrent calls, or calls
after 5.srcu_barrier().
The combination of srcu_barrier() and cleanup_srcu_struct() typically occurs on
the module exit path.
srcu_barrier() ensures that all previously inserted callbacks completeļ¼so
therefore, theoretically,
there shouldn't be any callbacks left to execute after we finish executing
srcu_barrier().
and of course, there are also shouldn't be any ongoing or newly started SRCU
grace period and the
WARN_ON() in cleanup_srcu_struct() is designed to detect it.
If when the cleanup_srcu_struct() detects an incomplete SRCU grace period or
any srcu callbacks
that have not yet been executed, this is a risk that needs to be reported, even
if the
cleanup_srcu_struct() can intercept it internally.
Therefore, we should investigate this issue to ensure that 5.srcu_barrier() can intercept the step 4 callback.
Thanks
Zqiang
Before 5. srcu_barrier() call, the 4. call_srcu() is finished and the
callback was enqueued, and srcu_barrier() did intercept it by appending
barrier cb after it. They can't be called concurrently because when
call_srcu() is called inside kvm_io_bus_register_dev(), the kvm
reference count is nonzero and kvm_destroy_vm() which calls
srcu_barrier() could not be started.
In my debugging, all five __free_bus() callbacks enqueued by
kvm_io_bus_register_dev() were invoked before the barrier cb, and only
then the barrier cb is called and srcu_barrier() returns. So all the
real callbacks were executed and practically there were no callbacks
left to execute when srcu_barrier() returns, but a stale n_cbs > 0 is
left because srcu_invoke_callbacks()'s invoking loop has not finished.
KVM logic correctly called srcu_barrier() and cleanup_srcu_struct()
without calling call_srcu() in between. In my opinion, the root causes
are as follows:
1) for timer_delete_sync(): the srcu_gp_end() which ended the last grace
period (from 4. ) arms sdp->delay_work to expire at jiffies + 1 even
though the invoke work (queued in 3. ) is already queued, and
cleanup_srcu_struct() runs before that one-jiffy timer expires, so
timer_delete_sync() cancels that pending timer and returns true.
2) for n_cbs: srcu_invoke_callbacks() calls rcu_segcblist_add_len(-len)
only at the end of the work item, after the barrier callback has been
invoked and srcu_barrier() has already returned, so
cleanup_srcu_struct() observes a stale n_cbs > 0 while the cblist is
physically empty.
Thanks
Sunho Park
