> > On 8/26/26 22:13, Zqiang wrote: > > > > > In this SRCU_SIZE_SMALL mode, when queue delayed work timer, the cblist is > > always no-empty. > > unless invoke start_poll_synchronize_srcu() to begin SRCU garce period, > > otherwise invoke call_srcu() or synchronize_srcu*() will insert callback. > > > Right, every grace period which armed the delay_work timer is started by the > real call_srcu(&kvm->srcu, &bus->rcu, __free_bus) in > kvm_io_bus_register_dev(). I missed one point: the invoke work which invoked > the barrier callbacks is not queued by the nearest end of grace period. It > was queued by the timer of a previous end of GP, and it started only after > the last GP had ended. Meanwhile the srcu_gp_end() from the last GP armed > another timer even though the work was already queued. The timeline is as > below: > > 1. call_srcu(&kvm->srcu, &bus->rcu, __free_bus) > 2. One end of GP comes, arms a timer. > 3. The timer is fired and an invoke work is queued to rcu_gp_wq. The timer is > disabled now. > 4. Another call_srcu(&kvm->srcu, &bus->rcu, __free_bus) > 5. srcu_barrier() is called and queues barrier callbacks, waits for > srcu_invoke_callbacks() to invoke them.
Based on your description below, the 5.srcu_barrier() did not intercept the callback of 4.call_srcu(), this means that 4.call_srcu() and 5.srcu_barrier() concurrent calls, or calls after 5.srcu_barrier(). The combination of srcu_barrier() and cleanup_srcu_struct() typically occurs on the module exit path. srcu_barrier() ensures that all previously inserted callbacks completeļ¼so therefore, theoretically, there shouldn't be any callbacks left to execute after we finish executing srcu_barrier(). and of course, there are also shouldn't be any ongoing or newly started SRCU grace period and the WARN_ON() in cleanup_srcu_struct() is designed to detect it. If when the cleanup_srcu_struct() detects an incomplete SRCU grace period or any srcu callbacks that have not yet been executed, this is a risk that needs to be reported, even if the cleanup_srcu_struct() can intercept it internally. Therefore, we should investigate this issue to ensure that 5.srcu_barrier() can intercept the step 4 callback. Thanks Zqiang > 6. The end of GP from step 4 comes, arms another timer. > 7. The invoke work queued in step 3 starts, srcu_invoke_callbacks() is > called. It starts invoking callbacks without subtracting the cblist len > field. It will call rcu_segcblist_add_len(&sdp->srcu_cblist, -len) after the > invoking loop is over. > When I debugged, there were five __free_bus(the real callbacks) and one > barrier callback, so the cblist len field was 6. > 8. Barrier callback is invoked, still the cblist len field is not subtracted > as srcu_invoke_callbacks()'s invoking loop is not over. > 9. srcu_barrier() wakes up by completion and cleanup_srcu_struct() is called > before the timer armed in step 6 expires. > At this point the cblist is physically empty (head == NULL, all seglen are 0) > as all six callbacks have already been invoked. Only the cblist len field is > stale(>0). > > Thanks > Sunho Park >
