Re: [libvirt] [PATCH] libxl: find virDomainObj in libxlDomainShutdownThread
Martin Kletzander wrote: > On Wed, Sep 21, 2016 at 03:29:49PM -0600, Jim Fehlig wrote: >> libxl events are delivered to libvirt via the libxlDomainEventHandler >> callback registered with libxl. Documenation in >> $xensrc/tools/libxl/libxl_event.h states that the callback "may occur >> on any thread in which the application calls libxl". This can result >> in deadlock since many of the libvirt callees of libxl hold a lock on >> the virDomainObj they are working on. When the callback is invoked, it >> attempts to find a virDomainObj corresponding to the domain ID provided >> by libxl. Searching the domain obj list results in locking each obj >> before checking if it is active, and its ID equals the requested ID. >> Deadlock is possible when attempting to lock an obj that is already >> locked further up the call stack. Indeed, Max Ustermann reported an >> instance of this deadlock >> >> https://www.redhat.com/archives/libvir-list/2015-November/msg00130.html >> >> Guido Rossmueller also recently stumbled across it >> >> https://www.redhat.com/archives/libvir-list/2016-September/msg00287.html >> >> Fix the deadlock by moving the lookup of virDomainObj to the >> libxlDomainShutdownThread. After this patch, libxl events are >> enqueued on the libvirt side and processed by dedicated thread, >> avoiding the described deadlock. >> > > Makes sense, passing locked object to another thread is not good, the > references match, so even though I cannot test this out, I'm fairly sure > when giving it an ACK. Thanks Martin! I finally got around to pushing this before the freeze. Regards, Jim -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH] libxl: find virDomainObj in libxlDomainShutdownThread
On Wed, Sep 21, 2016 at 03:29:49PM -0600, Jim Fehlig wrote: libxl events are delivered to libvirt via the libxlDomainEventHandler callback registered with libxl. Documenation in $xensrc/tools/libxl/libxl_event.h states that the callback "may occur on any thread in which the application calls libxl". This can result in deadlock since many of the libvirt callees of libxl hold a lock on the virDomainObj they are working on. When the callback is invoked, it attempts to find a virDomainObj corresponding to the domain ID provided by libxl. Searching the domain obj list results in locking each obj before checking if it is active, and its ID equals the requested ID. Deadlock is possible when attempting to lock an obj that is already locked further up the call stack. Indeed, Max Ustermann reported an instance of this deadlock https://www.redhat.com/archives/libvir-list/2015-November/msg00130.html Guido Rossmueller also recently stumbled across it https://www.redhat.com/archives/libvir-list/2016-September/msg00287.html Fix the deadlock by moving the lookup of virDomainObj to the libxlDomainShutdownThread. After this patch, libxl events are enqueued on the libvirt side and processed by dedicated thread, avoiding the described deadlock. Makes sense, passing locked object to another thread is not good, the references match, so even though I cannot test this out, I'm fairly sure when giving it an ACK. signature.asc Description: Digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list