On Tue, Oct 03, 2017 at 05:21:57PM -0500, Michael Roth wrote: > Quoting Michael Roth (2017-07-26 20:30:52) > > This series was motivated by the discussion in this thread: > > > > https://www.redhat.com/archives/libvir-list/2017-June/msg01370.html > > > > The issue this series addresses is that when libvirt unplugs a VFIO PCI > > device, > > it may attempt to bind the host device back to the host driver when QEMU > > emits > > the DEVICE_DELETED event for the corresponding vfio-pci device. However, the > > VFIO group FD is not actually cleaned up until vfio-pci device is > > *finalized* > > by QEMU, whereas the event is emitted earlier during device_unparent. > > Depending on the host device and how long certain operations like resetting > > the > > device might take, this can in result in libvirt trying to rebind the device > > back to the host while it is still in use by VFIO, leading to host crashes > > or > > other unexpected behavior. > > > > In particular, Mellanox CX4 adapters on PowerNV hosts might not be fully > > quiesced by vfio-pci's finalize() routine until up to 6s after the > > DEVICE_DELETED was emitted, leading to detach-device on the libvirt side > > pretty > > much always crashing the host. > > > > Implementing this change requires 2 prereqs to ensure the same information > > is > > available when the DEVICE_DELETED is finally emitted: > > > > 1) Storing the path in the composition patch, which is addressed by PATCH 1, > > which was plucked from another pending series from Greg Kurz: > > > > https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg07922.html > > > > since we are now "disconnected" at the time the event is emitted, and > > > > 2) Deferring qemu_opts_del of the DeviceState->QemuOpts till finalize, since > > that is where DeviceState->id is stored. This was actually how it was > > done in the past, so PATCH 2 simply reverts the change which moved it to > > device_unparent. > > > > From there it's just a mechanical move of the event from device_unparent to > > device_finalize. > > Ping. > > The situation has changed somewhat since original posting as Alex now > has a fix on the kernel side for the VFIO issue noted above: > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6586b561a91cd80a91c8f107ed0d144feb3eadc2 > > However, I think this series would still be useful for addressing the > issue for hosts using older kernels, and there seems to be general > interest from the libvirt side in aligning "DEVICE_DELETED" events > with the notion that QEMU is completely finished with a device. > > Patch 1/3 is also needed for another series that Greg is working on > and I don't want to hold that up for this series, so if it's > preferred that we just post that patch separately in the meantime > please let me know.
Yeah, I'd been meaning to ask you about this. I think it just got
lost in the rush of things for qemu-2.10. I think it would be worth
rebasing and reposting.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
signature.asc
Description: PGP signature
