On 13.02.26 22:35, [email protected] wrote: > From: Jan Kiszka <[email protected]> Sent: Thursday, February 12, 2026 > 8:06 AM >> >> On 09.02.26 19:25, Michael Kelley wrote: >>> From: Florian Bezdeka <[email protected]> Sent: Monday, February >>> 9, 2026 2:35 AM >>>> >>>> On Sat, 2026-02-07 at 01:30 +0000, Michael Kelley wrote: >>>> >>>> [snip] >>>>> >>>>> I've run your suggested experiment on an arm64 VM in the Azure cloud. My >>>>> kernel was linux-next 20260128. I set CONFIG_PREEMPT_RT=y and >>>>> CONFIG_PROVE_LOCKING=y, but did not add either of your two patches >>>>> (neither the storvsc driver patch nor the x86 VMBus interrupt handling >>>>> patch). >>>>> The VM comes up and runs, but with this warning during boot: >>>>> >>>>> [ 3.075604] hv_utils: Registering HyperV Utility Driver >>>>> [ 3.075636] hv_vmbus: registering driver hv_utils >>>>> [ 3.085920] ============================= >>>>> [ 3.088128] hv_vmbus: registering driver hv_netvsc >>>>> [ 3.091180] [ BUG: Invalid wait context ] >>>>> [ 3.093544] 6.19.0-rc7-next-20260128+ #3 Tainted: G E >>>>> [ 3.097582] ----------------------------- >>>>> [ 3.099899] systemd-udevd/284 is trying to lock: >>>>> [ 3.102568] ffff000100e24490 (&channel->sched_lock){....}-{3:3}, at: >>>>> vmbus_chan_sched+0x128/0x3b8 [hv_vmbus] >>>>> [ 3.108208] other info that might help us debug this: >>>>> [ 3.111454] context-{2:2} >>>>> [ 3.112987] 1 lock held by systemd-udevd/284: >>>>> [ 3.115626] #0: ffffd5cfc20bcc80 (rcu_read_lock){....}-{1:3}, at: >>>>> vmbus_chan_sched+0xcc/0x3b8 [hv_vmbus] >>>>> [ 3.121224] stack backtrace: >>>>> [ 3.122897] CPU: 0 UID: 0 PID: 284 Comm: systemd-udevd Tainted: G >>>>> E 6.19.0-rc7-next-20260128+ #3 PREEMPT_RT >>>>> [ 3.129631] Tainted: [E]=UNSIGNED_MODULE >>>>> [ 3.131946] Hardware name: Microsoft Corporation Virtual >>>>> Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 06/10/2025 >>>>> [ 3.138553] Call trace: >>>>> [ 3.140015] show_stack+0x20/0x38 (C) >>>>> [ 3.142137] dump_stack_lvl+0x9c/0x158 >>>>> [ 3.144340] dump_stack+0x18/0x28 >>>>> [ 3.146290] __lock_acquire+0x488/0x1e20 >>>>> [ 3.148569] lock_acquire+0x11c/0x388 >>>>> [ 3.150703] rt_spin_lock+0x54/0x230 >>>>> [ 3.152785] vmbus_chan_sched+0x128/0x3b8 [hv_vmbus] >>>>> [ 3.155611] vmbus_isr+0x34/0x80 [hv_vmbus] >>>>> [ 3.158093] vmbus_percpu_isr+0x18/0x30 [hv_vmbus] >>>>> [ 3.160848] handle_percpu_devid_irq+0xdc/0x348 >>>>> [ 3.163495] handle_irq_desc+0x48/0x68 >>>>> [ 3.165851] generic_handle_domain_irq+0x20/0x38 >>>>> [ 3.168664] gic_handle_irq+0x1dc/0x430 >>>>> [ 3.170868] call_on_irq_stack+0x30/0x70 >>>>> [ 3.173161] do_interrupt_handler+0x88/0xa0 >>>>> [ 3.175724] el1_interrupt+0x4c/0xb0 >>>>> [ 3.177855] el1h_64_irq_handler+0x18/0x28 >>>>> [ 3.180332] el1h_64_irq+0x84/0x88 >>>>> [ 3.182378] _raw_spin_unlock_irqrestore+0x4c/0xb0 (P) >>>>> [ 3.185493] rt_mutex_slowunlock+0x404/0x440 >>>>> [ 3.187951] rt_spin_unlock+0xb8/0x178 >>>>> [ 3.190394] kmem_cache_alloc_noprof+0xf0/0x4f8 >>>>> [ 3.193100] alloc_empty_file+0x64/0x148 >>>>> [ 3.195461] path_openat+0x58/0xaa0 >>>>> [ 3.197658] do_file_open+0xa0/0x140 >>>>> [ 3.199752] do_sys_openat2+0x190/0x278 >>>>> [ 3.202124] do_sys_open+0x60/0xb8 >>>>> [ 3.204047] __arm64_sys_openat+0x2c/0x48 >>>>> [ 3.206433] invoke_syscall+0x6c/0xf8 >>>>> [ 3.208519] el0_svc_common.constprop.0+0x48/0xf0 >>>>> [ 3.211050] do_el0_svc+0x24/0x38 >>>>> [ 3.212990] el0_svc+0x164/0x3c8 >>>>> [ 3.214842] el0t_64_sync_handler+0xd0/0xe8 >>>>> [ 3.217251] el0t_64_sync+0x1b0/0x1b8 >>>>> [ 3.219450] hv_utils: Heartbeat IC version 3.0 >>>>> [ 3.219471] hv_utils: Shutdown IC version 3.2 >>>>> [ 3.219844] hv_utils: TimeSync IC version 4.0 >>>> >>>> That matches with my expectation that the same problem exists on arm64. >>>> The patch from Jan addresses that issue for x86 (only, so far) as we do >>>> not have a working test environment for arm64 yet. >>> >>> OK. I had understood Jan's earlier comments to mean that the VMBus >>> interrupt problem was implicitly solved on arm64 because of VMBus using >>> a standard Linux IRQ on arm64. But evidently that's not the case. So my >>> earlier comment stands: The code changes should go into the architecture >>> independent portion of the VMBus driver, and not under arch/x86. I >>> can probably work with you to test on arm64 if need be. >>> >> >> I can move the code, sure, but I still haven't understood what >> invalidates my assumptions (beside what you observed). vmbus_drv calls >> request_percpu_irq, and that is - as far as I can see - not injecting >> IRQF_NO_THREAD. Any explanations welcome. > > I haven't setup detailed debugging on arm64 yet, but in prep for that > I went looking at the places in the kernel IRQ handling where > IRQF_NO_THREAD influences behavior. The key function appears to be > irq_setup_forced_threading(). This function first checks force_irqthreads(), > which will be "true" when PREEMPT_RT is set. The function then checks > the IRQF_NO_THREAD flag and the IRQF_PERCPU flag. From what I can > see, the IRQF_PERCPU flag is treated like the IRQF_NO_THREAD flag, and > causes forced threading to *not* be done. So the behavior ends up being > the same as when PREEMPT_RT is not set. > > Since the VMBus interrupt is a per-cpu interrupt, forced threading is not > done. In that case, the stack trace I reported makes sense. Take a look at > the code and see if you agree.
Indeed, missed the IRQF_PERCPU impact on auto-threading. I'll rework my patch to perform the transition arch-independently. Thanks, Jan -- Siemens AG, Foundational Technologies Linux Expert Center
