st 26. 8. 2026 v 23:21 odesÃlatel Crystal Wood <[email protected]> napsal:
>
> On Wed, 2026-08-26 at 15:31 -0400, Steven Rostedt wrote:
> > On Mon, 24 Aug 2026 21:30:37 +0000
> > [email protected] wrote:
> >
> > > [Severity: High]
> > > Can holding osn_var->lock while calling kthread_stop() cause a deadlock?
> > >
> > > kthread_stop() blocks waiting for the target kthread to exit. However, if
> > > the kthread is currently handling a migration request in
> > > osnoise_migration_pending() (shown slightly earlier in the diff), it will
> > > attempt to acquire this same osn_var->lock before it can clear its state
> > > and
> > > exit:
> >
> > This code has a lot of nastiness in the locking with regard to the thread.
> > I haven't looked deeply at your patch, but the fact that Sashiko pointed
> > out a possible deadlock I think you may be hitting the mess I was hitting.
> >
> > Have you looked at this thread:
> > https://lore.kernel.org/all/[email protected]/
>
> That thread is about user fd sync, which is what this patch is trying to
> address.
>
> This particular deadlock is with kernel threads, in a corner case of
> getting migrated even though we try to pin them to one cpu. We can get
> rid of this migration code and just have stop_kthread() take care of it,
> adding a get/put_task_struct() so that it's OK for the thread to die
> early. This way we can also handle any other abnormal thread exits.
>
> I'll respond to the rest of the Sashiko comments soon.
>
> >
> > It has some tests I ran along with tracing that detected issues. Of course,
> > everything needs to be run with lockdep enabled too.
>
> I did run with lockdep (and some custom hacky state tracking) but
> forcing migration is a hole in my test coverage.
>
> Is there a particular test you're suggesting?
>
> -Crystal
The race window is practically really small. I had success of
reproducing the migration deadlock in a virtual machine with a bash
script [1] that deliberately slows down the kernel/osnoise tracer with
perf, ftrace, and bpftrace hooks:
[root@cs9 tglozar]#
/home/tglozar/dev/linux/tools/tracing/rtla/osnoise_migration_race.sh
13 10000
cpu=13 mask=0,1,2,3,4,5,6,7,8,9,10,11,12 iters=10000 stop=nop
iter 1 ok (kpid 265080)
iter 2 ok (kpid 265086)
...
iter 422 ok (kpid 267814)
*** hang iter 423 kthread=267821 affiner=267823 stopper=267827
cat /proc/267821/stack /proc/267827/stack
do not rmdir instances/ or write current_tracer/online; reboot
[root@cs9 tglozar]# cat /proc/267821/stack; echo -------; cat /proc/267827/stack
[<0>] timerlat_main+0x266/0x3f0
[<0>] kthread+0xe6/0x120
[<0>] ret_from_fork+0x1be/0x250
[<0>] ret_from_fork_asm+0x1a/0x30
-------
[<0>] kthread_stop+0x6b/0x180
[<0>] stop_kthread+0x86/0xc0
[<0>] stop_per_cpu_kthreads+0xf/0x40
[<0>] osnoise_workload_stop.part.0+0xc/0xa0
[<0>] timerlat_tracer_reset+0x9/0x40
[<0>] tracing_set_tracer+0x14d/0x300
[<0>] tracing_set_trace_write+0x7f/0xe0
[<0>] vfs_write+0xf8/0x410
[<0>] ksys_write+0x65/0xe0
[<0>] do_syscall_64+0xc1/0x460
[<0>] entry_SYSCALL_64_after_hwframe+0x76/0x7e
[1] https://gitlab.com/-/snippets/6045758
Note that any path that is calling kthread_stop() can trigger the
deadlock (together with the migration), that is, both making the CPU
offline and stopping the osnoise tracer. Sashiko flagged the former,
my reproducer used the latter (the script can do both).
Tomas
