> > > > > On 8/27/26 08:53, Zqiang wrote: > > > > > > Based on your description below, the 5.srcu_barrier() did not intercept > > the callback of 4.call_srcu(), > > this means that 4.call_srcu() and 5.srcu_barrier() concurrent calls, or > > calls after 5.srcu_barrier(). > > The combination of srcu_barrier() and cleanup_srcu_struct() typically > > occurs on the module exit path. > > srcu_barrier() ensures that all previously inserted callbacks completeļ¼so > > therefore, theoretically, > > there shouldn't be any callbacks left to execute after we finish executing > > srcu_barrier(). > > and of course, there are also shouldn't be any ongoing or newly started > > SRCU grace period and the > > WARN_ON() in cleanup_srcu_struct() is designed to detect it. > > If when the cleanup_srcu_struct() detects an incomplete SRCU grace period > > or any srcu callbacks > > that have not yet been executed, this is a risk that needs to be reported, > > even if the > > cleanup_srcu_struct() can intercept it internally. > > > Therefore, we should investigate this issue to ensure that > > 5.srcu_barrier() can intercept the step 4 callback. > > Thanks > > Zqiang > > > > Before 5. srcu_barrier() call, the 4. call_srcu() is finished and the > > callback was enqueued, and srcu_barrier() did intercept it by appending > > barrier cb after it. They can't be called concurrently because when > > call_srcu() is called inside kvm_io_bus_register_dev(), the kvm reference > > count is nonzero and kvm_destroy_vm() which calls srcu_barrier() could not > > be started. > > > > Which tree is your test based on? (rcu tree or linux-next tree) > > > > The step 4 call_srcu() is finished and then the step5 call srcu_barrier(), > > it failed to intercept it? > > (theoretically, this shouldn't happen) > > > Sorry, here I missed. > > > > > In my debugging, all five __free_bus() callbacks enqueued by > > kvm_io_bus_register_dev() were invoked before the barrier cb, and only then > > the barrier cb is called and srcu_barrier() returns. So all the real > > callbacks were executed and practically there were no callbacks left to > > execute when srcu_barrier() returns, but a stale n_cbs > 0 is left because > > srcu_invoke_callbacks()'s invoking loop has not finished. > > > > KVM logic correctly called srcu_barrier() and cleanup_srcu_struct() > > without calling call_srcu() in between. In my opinion, the root causes are > > as follows: > > > > 1) for timer_delete_sync(): the srcu_gp_end() which ended the last grace > > period (from 4. ) arms sdp->delay_work to expire at jiffies + 1 even though > > the invoke work (queued in 3. ) is already queued, and > > cleanup_srcu_struct() runs before that one-jiffy timer expires, so > > timer_delete_sync() cancels that pending timer and returns true. > > > The step5 srcu_barrier() has been intercept the srcu callback which by step4 > call_srcu() insert, > so at here, the srcu_barrier should not be return, and the step4 srcu > callback should not be run. > the cleanup_srcu_struct() also should not be called. > > Thanks > Zqiang > > > > > 2) for n_cbs: srcu_invoke_callbacks() calls rcu_segcblist_add_len(-len) > > only at the end of the work item, after the barrier callback has been > > invoked and srcu_barrier() has already returned, so cleanup_srcu_struct() > > observes a stale n_cbs > 0 while the cblist is physically empty.
Now, I am trying to rephrase your problem in the following way: 1- the step1 call_srcu() trigger SRCU grace period has been end and queue sdp->work. 2- the step4 queue srcu callback has been intercepted by step5 srcu_barrier() and start a new SRCU grace period. 3- this new SRCU grace period end, and invoke rcu_seq_end(&sup->srcu_gp_seq), queue a timer. 4- the sdp->work begin run, and call srcu_segcblist_advance() with current sup->srcu_gp_seq. so the step4's srcu callback and step5's barrier calback both to be run. all callback finished, but the len not yet update. 5- the srcu_barrier() return, begin call srcu_clean_up(), and then the srcu_clean_up find the a timer is still pending and the cb_len also not be zero, trigger WARN_ON() right? Thanks Zqiang > > > > because you say the 5.srcu_barrier() can not intercept setp4 call_srcu() > > enqueue callback func, > > that means that the step4 srcu callback shuold not be run, the cblist also > > not empty. > > > > Thanks > > Zqiang > > > > > > Thanks > > Sunho Park > > >
