Hi Paul, thanks a lot! Sounds good. Will try to confirm your findings ASAP Regards Bernhard > Hi again, > > First the good news: I found a way to make live migrations work again. > > As quoted below, libvirt changes the ownership of the guest image, > unless it detects that the image is on a shared filesystem. After > looking at the code for libvirt, they have code to detect NFS, GFS2 and > SMB/CIFS, but not Gluster. As libvirt does not detect that the storage > is on a shared file system, the originating host will perform a chown > back to root:root at the end of a successful migration, whereas the > destination host will do a chown to libvirt-qemu:kvm. This is in fact a > race condition, so the difference in behaviour between 3.4.0 and 3.4.1 > could be down to timing differences. > > Workaround: stop your guests, then stop libvirt, and edit > /etc/libvirt/qemu.conf - this contains a commented out entry > 'dynamic_ownership=1', which is the default. Change this to 0, and > remove the comment. Then do a chown to libvirt-qemu:kvm for all your > stopped images. Then you can start the service libvirt-bin again, and > bring up the guests. Repeat on the other half of your cluster, and test > a live migration. For me, they work again. > > The workaround seems fine with me, but now you have to take care of > properly setting the ownership of a guest image yourself (presumably > only once when you create it). > > Other possible solutions: > > JoeJulian suggested using libgfapi, giving libvirt direct access without > having to go through the filesystem. This is the preferred setup for > libvirt+gluster and should also result in better I/O performance. I > haven't tested this yet, but it's high on my to-do list. > > Submit a patch to libvirt so it can detect that the filesystem is > Gluster. statfs() will only show 'FUSE", but we could then use getxattr > to see if there is a gluster-specific attribute set (suggested by > kkeithley). This could be trusted.glusterfs.volume-id, e.g. > > Regards, Paul Boven. > > > On 01/28/2014 01:57 PM, Paul Boven wrote: > > Hi everyone, > > > > On the libvirt Wiki, I found the text below which might well apply to > > our live-migration issue: > > > > "The directory used for storing disk images has to be mounted from > > shared storage on both hosts. Otherwise, the domain may lose access to > > its disk images during migration because source libvirtd may change the > > owner, permissions, and SELinux labels on the disk images once it > > successfully migrates the domain to its destination. Libvirt avoids > > doing such things if it detects that the disk images are mounted from a > > shared storage. " > > > > So perhaps libvirtd fails to recognize that it is on shared storage, and > > it is the originating libvirt that throws a wrench in the wheels by > > changing the ownership? > > > > http://wiki.libvirt.org/page/Migration_fails_because_disk_image_cannot_be_found > > > > > > Regards, Paul Boven.
> > -- > Paul Boven <> [email protected]> > +31 (0)521-596547 > Unix/Linux/Networking specialist > Joint Institute for VLBI in Europe - www.jive.nl > VLBI - It's a fringe science > _______________________________________________ > Gluster-users mailing list > [email protected] > http://supercolony.gluster.org/mailman/listinfo/gluster-users -- Bernhard Glomm IT Administration Phone: +49 (30) 86880 134 Fax: +49 (30) 86880 100 Skype: bernhard.glomm.ecologic Ecologic Institut gemeinnützige GmbH | Pfalzburger Str. 43/44 | 10717 Berlin | Germany GF: R. Andreas Kraemer | AG: Charlottenburg HRB 57947 | USt/VAT-IdNr.: DE811963464 Ecologic™ is a Trade Mark (TM) of Ecologic Institut gemeinnützige GmbH
_______________________________________________ Gluster-users mailing list [email protected] http://supercolony.gluster.org/mailman/listinfo/gluster-users
