On Thu, Jul 02, 2026 at 09:30:26PM +0200, David Hildenbrand (Arm) wrote:
> On 7/2/26 19:50, Denis V. Lunev wrote:
> > On 6/24/26 16:08, Denis V. Lunev wrote:
> >> This email originated from an IP that might not be authorized by the
> >> domain it was sent from.
> >> Do not click links or open attachments unless it is an email you expected
> >> to receive.
> >> Since commit 8bd2fa086a04 ("virtio: break and reset virtio devices on
> >> device_shutdown()") the virtio bus breaks and resets every virtio device
> >> during device_shutdown(), i.e. on reboot and kexec. virtio_balloon has no
> >> .shutdown of its own, so that generic path runs while the balloon's
> >> asynchronous work is still armed: the free page reporting worker, the
> >> inflate/deflate and stats workers, the OOM notifier and the free page
> >> shrinker.
> >>
> >> Once the device has been broken, virtqueue_add_inbuf() in
> >> virtballoon_free_page_report() returns -EIO and trips its WARN_ON_ONCE().
> >> On a kernel booted with panic_on_warn that turns an ordinary reboot into a
> >> fatal panic in the middle of device_shutdown(), so the machine never
> >> reaches the new kernel. The inflate/deflate and OOM paths do not warn but
> >> are no better off: they call wait_event(vb->acked, ...) and would block
> >> forever on a queue that can no longer complete.
> >>
> >> This was hit in the field as an intermittent failure of a virtualization
> >> cluster upgrade: guest storage nodes were rebooted via kexec into the new
> >> kernel, and the ones whose free page reporting happened to run during
> >> device_shutdown() panicked (the guests run with panic_on_warn) and never
> >> came back, stalling the rolling upgrade. The crash dump showed the WARN at
> >> virtio_balloon.c:216 in a page_reporting kworker, with all the balloon
> >> virtqueues already broken.
> >>
> >> Validated by churning balloon inflate/deflate from the host while
> >> kexec-rebooting the guest in a loop under panic_on_warn: the unpatched
> >> kernel reproduces the WARN within a couple of cycles, while the patched
> >> kernel survives many consecutive kexec cycles cleanly (12/12 in the final
> >> run, 0 WARNs). checkpatch is clean across the series.
> >>
> >> Changes in v2:
> >> - Add a virtio_device_shutdown() core helper and call it from the balloon
> >> .shutdown handler instead of open-coding break + synchronize_cbs + reset
> >> (David Hildenbrand).
> >> - New patch: make tell_host() warn and bail instead of hanging if a buffer
> >> add ever fails (David Hildenbrand); kept as a separate patch
> >> (Michael S. Tsirkin).
> >>
> >> v1: https://lore.kernel.org/all/[email protected]
> >>
> >> Denis V. Lunev (4):
> >> virtio: add virtio_device_shutdown() helper
> >> virtio_balloon: factor out virtballoon_quiesce()
> >> virtio_balloon: quiesce balloon work before device shutdown
> >> virtio_balloon: warn on failed buffer add in tell_host()
> >>
> >> drivers/virtio/virtio.c | 41 ++++++++++++++++++++++-----------
> >> drivers/virtio/virtio_balloon.c | 40 ++++++++++++++++++++++++--------
> >> include/linux/virtio.h | 1 +
> >> 3 files changed, 59 insertions(+), 23 deletions(-)
> >>
> > Hi, David!
>
> Hi! :)
>
> >
> > Is this good to go in? I have not seen the confirmation that the
> > series is taken into your tree.
>
> I don't have a tree (yet), and once I have one it will likely be more mm
> focused :)
>
> @MST, I think this is good to go!
>
> --
> Cheers,
>
> David
Indeed, it's just a bugfix and I'm trying to get features into qemu
now before their freeze. I'll work on linux end of next week.
