Tested on Meteor Lake (Core Ultra 5 125H, vpu_37xx firmware 20260219)
on top of v7.4-rc1:

- kexec from a 7.0 kernel and from 7.4 with the NPU runtime-suspended:
  both runs hit ATS_ERR during probe, recovery_work was not queued,
  probe failed once with -ETIMEDOUT, and there was no oops
- FLR plus rebind brings the NPU back
- debugfs force_recovery after a successful probe recovers the NPU

I couldn't hit an error IRQ after ivpu_boot() succeeds, so the deferred
recovery in ivpu_pm_enable() isn't covered by this.

Tested-by: Daksh Sahni <[email protected]>

On Tue, Nov 10, 2026 at 2:03 PM Karol Wachowski
<[email protected]> wrote:
>
> On 07-Nov-26 18:42, Daksh Sahni wrote:
> > If the NPU raises an error interrupt while ivpu_dev_init() boots the
> > firmware, the Buttress, MMU or WDT IRQ handler calls
> > ivpu_pm_trigger_recovery(), which queues recovery_work. When ivpu_boot()
> > then fails, the err_ipc_fini path tears the device down without
> > disabling recovery_work and probe returns an error. devres unmaps the
> > register BARs while the work is still queued or running, and the work
> > faults on the first register access:
> >
> >    BUG: unable to handle page fault for address: ffffcfc6680200f4
> >    Workqueue: events_unbound ivpu_pm_recovery_work [intel_vpu]
> >    RIP: 0010:ivpu_hw_ip_ipc_rx_addr_get+0xc/0xb0 [intel_vpu]
> >    Call Trace:
> >     ivpu_wait_for_ready+0x155/0x220 [intel_vpu]
> >     ivpu_boot+0x9e/0x270 [intel_vpu]
> >     ivpu_resume+0xa4/0xd0 [intel_vpu]
> >     ivpu_pm_reset_complete+0x2c/0x90 [intel_vpu]
> >     ivpu_pm_recovery_work+0xf6/0x140 [intel_vpu]
> >     process_one_work+0x1da/0x3c0
> >     worker_thread+0x1af/0x340
> >     kthread+0xfa/0x130
> >     ret_from_fork+0x2b9/0x380
> >     ret_from_fork_asm+0x1a/0x30
> >
> > The work dies between pm_runtime_disable() in ivpu_pm_reset_begin() and
> > pm_runtime_enable() in ivpu_pm_reset_complete(), so the PCI device is
> > also left with runtime PM disabled and never autosuspends after a later
> > rebind.
> >
> > Commit 5eaa49741119 ("accel/ivpu: Prevent recovery invocation during
> > probe and resume") stopped IPC failures during probe from triggering
> > recovery, but the IRQ handlers still can.
> >
> > Call ivpu_pm_disable_recovery() at the start of the err_ipc_fini path,
> > as ivpu_dev_fini() already does, so the work is flushed and cannot be
> > queued again before the device is torn down.
> >
> > Reproduced on Meteor Lake (Core Ultra 5 125H, vpu_37xx firmware
> > 20260219) by kexec'ing with the NPU runtime-suspended: probe in the new
> > kernel gets an ATS_ERR Buttress IRQ during firmware boot. With this
> > change probe still fails with -ECANCELED, there is no oops, and an FLR
> > followed by a rebind brings the NPU back.
> >
> > Fixes: 852be13f3bd3 ("accel/ivpu: Add PM support")
> > Cc: [email protected]
> > Signed-off-by: Daksh Sahni <[email protected]>
>
>   Hi Daksh,
>
>   Thanks for the report and the thorough analysis, it's correct. An error
>   IRQ that fires once ivpu_boot() has enabled interrupts can queue
>   recovery_work during probe, and nothing on the probe error path stops
>   it.
>
>   As Sashiko pointed out, disabling the work on err_ipc_fini comes too
>   late. By then recovery_work may already have run concurrently with
>   ivpu_boot(). It can also race with a probe that succeeds, which you saw
>   yourself in one of your runs.
>
>   I'll send a v2 that addresses this along the lines you offered and
>   Sashiko suggested. recovery_work stays disabled for the whole of probe
>   and is enabled in ivpu_pm_enable(). A recovery requested during probe
>   is not lost. If ivpu_boot() fails, probe unwinds with nothing pending.
>
>   If  probe succeeds, ivpu_pm_enable() runs the deferred recovery and
>   waits for it before issuing the initial autosuspend request. The NPU is
>   then reset before first use, and the autosuspend no longer lands while
>   runtime PM is disabled, which covers your second note too. The patch
>   will keep your Reported-by.
>
>   I'm running it through our internal CI first and will post it once that
>   passes. When it's out, could you test it on your Meteor Lake kexec
>   setup and reply with your Tested-by? Once I have it, I'll go ahead
>   and merge.
>
>   Thanks,
>   Karol
> > ---
> >
> > Notes:
> >      Tested on v7.4-rc1 (d48723711bae).
> >
> >      Two things this patch does not address:
> >
> >      The trigger is kexec with the NPU runtime-suspended in the old kernel.
> >      The driver has no .shutdown hook, so the new kernel's probe sees stale
> >      NPU state and gets ATS_ERR. A cold boot doesn't hit it. Whether that
> >      wants a shutdown hook or a reset at probe seems like your call.
> >
> >      In one of four runs that hit ATS_ERR, probe succeeded while
> >      recovery_work was running concurrently. Probe returned 0, but the
> >      autosuspend request from ivpu_pm_enable() landed while recovery had
> >      runtime PM disabled, so the NPU stayed active until first use. Not
> >      allowing IRQ-triggered recovery until ivpu_pm_enable() would cover both
> >      cases; I can send a v2 that does that if you prefer.
> >
> >   drivers/accel/ivpu/ivpu_drv.c | 1 +
> >   1 file changed, 1 insertion(+)
> >
> > diff --git a/drivers/accel/ivpu/ivpu_drv.c b/drivers/accel/ivpu/ivpu_drv.c
> > index 247ed6b2ff62..4fcccf78ec5d 100644
> > --- a/drivers/accel/ivpu/ivpu_drv.c
> > +++ b/drivers/accel/ivpu/ivpu_drv.c
> > @@ -852,6 +852,7 @@ static int ivpu_dev_init(struct ivpu_device *vdev)
> >       return 0;
> >
> >   err_ipc_fini:
> > +     ivpu_pm_disable_recovery(vdev);
> >       ivpu_ipc_fini(vdev);
> >   err_fw_fini:
> >       ivpu_fw_fini(vdev);
>

Reply via email to