[libvirt] libxl: Issues with virt-manager when used to manager Xen domains
This started off with some regression testing after going forward to Xen-4.4. We currently would pair that with a libvirt version 1.2.2 and right now operations through virsh seem to be working (mostly) well. But when using virt-manager (not the most up-to-date versions but some combinations that quite likely will occur (0.9.1 and 0.9.5)) there are some issues. Two symptoms seem to be caused by the same underlying problem which is caused by the way virt-manager interacts with libvirt. That seems to be that virt-manager acquires a reference to virDomainPtr only once and then uses that for subsequent call. This is a problem because both virDomainPtr object and virDomainObjPtr objects contains a domid but only the latter gets updated. So both, creating a new guest (which must be a define step and then use the handle to start the guest) as well as rebooting a guest cause problems for virt-manager. Mainly because of libxlDomainGetInfo() which retrieves the virDomainObjPtr for a given virDommainPtr. Then checks the domain object to be active or not but uses the domid from the domain to call into libxl: if (!(vm = libxlDomObjFromDomain(dom))) goto cleanup; ... if (!virDomainObjIsActive(vm)) { info-cpuTime = 0; info-memory = vm-def-mem.cur_balloon; info-maxMem = vm-def-mem.max_balloon; } else { if (libxl_domain_info(priv-ctx, d_info, dom-id) != 0) { This fails either with dom-id being -1 (still) or using the old id from before the reboot. A dirty hack seems to avoid this: static virDomainObjPtr libxlDomObjFromDomain(virDomainPtr dom) { virDomainObjPtr vm; libxlDriverPrivatePtr driver = dom-conn-privateData; char uuidstr[VIR_UUID_STRING_BUFLEN]; vm = virDomainObjListFindByUUID(driver-domains, dom-uuid); if (!vm) { virUUIDFormat(dom-uuid, uuidstr); virReportError(VIR_ERR_NO_DOMAIN, _(no domain with matching uuid '%s' (%s)), uuidstr, dom-name); return NULL; - } + } else { + if (dom-id != vm-def-id) { + VIR_WARN(libxlDomObjFromDomain: domid changed (%d-%d), +dom-id, vm-def-id); + dom-id = vm-def-id; + } + } return vm; } Not really sure this is the right way to go. Also because that warning above happens a lot more (repeatedly) than I would have expected. This may or may not be related to the next question. Not a problem but more a confusion on my side: virGetDomain() has commentary that says it will return either a pointer to an existing object or one to a new one. Its just, I cannot see that done in the code. Is that a future plan or was that way but was removed? I must admit I have not looked deeper into that but one explanation of the repeated domid mismatch would be if the virDomainPtr that virt-manager uses to communicate with libvirt would be cloned before doing things inside libvirt. And holding that handle for so long and not obtaining a fresh one through the various lookup interfaces each time could be a mistake of virt-manager. Unfortunately one that needs to be handled in some way. -Stefan signature.asc Description: OpenPGP digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] libxl: Issues with virt-manager when used to manager Xen domains
On Tue, Mar 25, 2014 at 04:22:54PM +0100, Stefan Bader wrote: This started off with some regression testing after going forward to Xen-4.4. We currently would pair that with a libvirt version 1.2.2 and right now operations through virsh seem to be working (mostly) well. But when using virt-manager (not the most up-to-date versions but some combinations that quite likely will occur (0.9.1 and 0.9.5)) there are some issues. Two symptoms seem to be caused by the same underlying problem which is caused by the way virt-manager interacts with libvirt. That seems to be that virt-manager acquires a reference to virDomainPtr only once and then uses that for subsequent call. This is a problem because both virDomainPtr object and virDomainObjPtr objects contains a domid but only the latter gets updated. So both, creating a new guest (which must be a define step and then use the handle to start the guest) as well as rebooting a guest cause problems for virt-manager. Mainly because of libxlDomainGetInfo() which retrieves the virDomainObjPtr for a given virDommainPtr. Then checks the domain object to be active or not but uses the domid from the domain to call into libxl: if (!(vm = libxlDomObjFromDomain(dom))) goto cleanup; ... if (!virDomainObjIsActive(vm)) { info-cpuTime = 0; info-memory = vm-def-mem.cur_balloon; info-maxMem = vm-def-mem.max_balloon; } else { if (libxl_domain_info(priv-ctx, d_info, dom-id) != 0) { This fails either with dom-id being -1 (still) or using the old id from before the reboot. A dirty hack seems to avoid this: Yep, this is a pretty commonly hit problem. The solution is to *never* access dom-id in the driver code at all. eg In this code snippet above you could use vm-def-id instead which should be up to date. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] libxl: Issues with virt-manager when used to manager Xen domains
On 25.03.2014 16:36, Daniel P. Berrange wrote: On Tue, Mar 25, 2014 at 04:22:54PM +0100, Stefan Bader wrote: This started off with some regression testing after going forward to Xen-4.4. We currently would pair that with a libvirt version 1.2.2 and right now operations through virsh seem to be working (mostly) well. But when using virt-manager (not the most up-to-date versions but some combinations that quite likely will occur (0.9.1 and 0.9.5)) there are some issues. Two symptoms seem to be caused by the same underlying problem which is caused by the way virt-manager interacts with libvirt. That seems to be that virt-manager acquires a reference to virDomainPtr only once and then uses that for subsequent call. This is a problem because both virDomainPtr object and virDomainObjPtr objects contains a domid but only the latter gets updated. So both, creating a new guest (which must be a define step and then use the handle to start the guest) as well as rebooting a guest cause problems for virt-manager. Mainly because of libxlDomainGetInfo() which retrieves the virDomainObjPtr for a given virDommainPtr. Then checks the domain object to be active or not but uses the domid from the domain to call into libxl: if (!(vm = libxlDomObjFromDomain(dom))) goto cleanup; ... if (!virDomainObjIsActive(vm)) { info-cpuTime = 0; info-memory = vm-def-mem.cur_balloon; info-maxMem = vm-def-mem.max_balloon; } else { if (libxl_domain_info(priv-ctx, d_info, dom-id) != 0) { This fails either with dom-id being -1 (still) or using the old id from before the reboot. A dirty hack seems to avoid this: Yep, this is a pretty commonly hit problem. The solution is to *never* access dom-id in the driver code at all. eg In this code snippet above you could use vm-def-id instead which should be up to date. OK, it just seems for something that should not be done, it is done plenty of times in the libxl_driver codebase right now. At least pause and unpause seem to do the same... -Stefan Regards, Daniel signature.asc Description: OpenPGP digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] libxl: Issues with virt-manager when used to manager Xen domains
On Tue, Mar 25, 2014 at 04:42:25PM +0100, Stefan Bader wrote: On 25.03.2014 16:36, Daniel P. Berrange wrote: On Tue, Mar 25, 2014 at 04:22:54PM +0100, Stefan Bader wrote: This started off with some regression testing after going forward to Xen-4.4. We currently would pair that with a libvirt version 1.2.2 and right now operations through virsh seem to be working (mostly) well. But when using virt-manager (not the most up-to-date versions but some combinations that quite likely will occur (0.9.1 and 0.9.5)) there are some issues. Two symptoms seem to be caused by the same underlying problem which is caused by the way virt-manager interacts with libvirt. That seems to be that virt-manager acquires a reference to virDomainPtr only once and then uses that for subsequent call. This is a problem because both virDomainPtr object and virDomainObjPtr objects contains a domid but only the latter gets updated. So both, creating a new guest (which must be a define step and then use the handle to start the guest) as well as rebooting a guest cause problems for virt-manager. Mainly because of libxlDomainGetInfo() which retrieves the virDomainObjPtr for a given virDommainPtr. Then checks the domain object to be active or not but uses the domid from the domain to call into libxl: if (!(vm = libxlDomObjFromDomain(dom))) goto cleanup; ... if (!virDomainObjIsActive(vm)) { info-cpuTime = 0; info-memory = vm-def-mem.cur_balloon; info-maxMem = vm-def-mem.max_balloon; } else { if (libxl_domain_info(priv-ctx, d_info, dom-id) != 0) { This fails either with dom-id being -1 (still) or using the old id from before the reboot. A dirty hack seems to avoid this: Yep, this is a pretty commonly hit problem. The solution is to *never* access dom-id in the driver code at all. eg In this code snippet above you could use vm-def-id instead which should be up to date. OK, it just seems for something that should not be done, it is done plenty of times in the libxl_driver codebase right now. At least pause and unpause seem to do the same... I expect those pretty much all need to be fixed Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] libxl: Issues with virt-manager when used to manager Xen domains
On 25.03.2014 16:46, Daniel P. Berrange wrote: On Tue, Mar 25, 2014 at 04:42:25PM +0100, Stefan Bader wrote: On 25.03.2014 16:36, Daniel P. Berrange wrote: On Tue, Mar 25, 2014 at 04:22:54PM +0100, Stefan Bader wrote: This started off with some regression testing after going forward to Xen-4.4. We currently would pair that with a libvirt version 1.2.2 and right now operations through virsh seem to be working (mostly) well. But when using virt-manager (not the most up-to-date versions but some combinations that quite likely will occur (0.9.1 and 0.9.5)) there are some issues. Two symptoms seem to be caused by the same underlying problem which is caused by the way virt-manager interacts with libvirt. That seems to be that virt-manager acquires a reference to virDomainPtr only once and then uses that for subsequent call. This is a problem because both virDomainPtr object and virDomainObjPtr objects contains a domid but only the latter gets updated. So both, creating a new guest (which must be a define step and then use the handle to start the guest) as well as rebooting a guest cause problems for virt-manager. Mainly because of libxlDomainGetInfo() which retrieves the virDomainObjPtr for a given virDommainPtr. Then checks the domain object to be active or not but uses the domid from the domain to call into libxl: if (!(vm = libxlDomObjFromDomain(dom))) goto cleanup; ... if (!virDomainObjIsActive(vm)) { info-cpuTime = 0; info-memory = vm-def-mem.cur_balloon; info-maxMem = vm-def-mem.max_balloon; } else { if (libxl_domain_info(priv-ctx, d_info, dom-id) != 0) { This fails either with dom-id being -1 (still) or using the old id from before the reboot. A dirty hack seems to avoid this: Yep, this is a pretty commonly hit problem. The solution is to *never* access dom-id in the driver code at all. eg In this code snippet above you could use vm-def-id instead which should be up to date. OK, it just seems for something that should not be done, it is done plenty of times in the libxl_driver codebase right now. At least pause and unpause seem to do the same... I expect those pretty much all need to be fixed All-right, thanks for confirming. :) Will work on that. -Stefan Regards, Daniel signature.asc Description: OpenPGP digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list