Re: [openstack-dev] [nova] correct API for getting image metadata for an instance ?
On 28 May 2015 at 19:26, Vladik Romanovsky vladik.romanov...@enovance.com wrote: As part of the work to object-ify the image metadata dicts, I'm looking at the current way the libvirt driver fetches image metadata for an instance, in cases where the compute manager hasn't already passed it into the virt driver API. I see 2 methods that libvirt uses to get the image metadata - nova.utils.get_image_from_system_metadata(instance.system_metadata) It takes the system metadata stored against the instance and turns it into image metadata. - nova.compute.utils.get_image_metadata(context, image_api, instance.image_ref, instance) This tries to get metadata from the image api and turns this into system metadata It then gets system metadata from the instance and merges it from the data from the image It then calls nova.utils.get_image_from_system_metadata() IIUC, any changes against the image will override what is stored against the instance IIUC, when an instance is booted, the image metadata should be saved against the instance. So I'm wondering why we need to have code in compute.utils that merges back in the image metadata each time ? Is this intentional so that we pull in latest changes from the image, to override what's previously saved on the instance ? If so, then it seems that we should have been consistent in using the compute_utils get_image_metadata() API everywhere. It seems wrong though to pull in the latest metadata from the image. The libvirt driver makes various decisions at boot time about how to configure the guest based on the metadata. When we later do changes to that guest (snapshot, hotplug, etc, etc) we *must* use exactly the same image metadata we had at boot time, otherwise decisions we make will be inconsistent with how the guest is currently configured. eg if you set hw_disk_bus=virtio at boot time, and then later change the image to use hw_disk_bus=scsi, and then try to hotplug a new drive on the guest, we *must* operate wrt hw_disk_bus=virtio because the guest will not have any scsi bus present. I agree, as well, that we should use the system_metadate instead of getting the latest from Glance. I just wish there would be an easy way to edit it, in order to update some keys such as video driver, watchdog action, nic driver.. etc, so it would be picked up on hard reboot, for example. We are trying to build a plan to deal with all those sorts of changes. The first step is trying to document what we are up to right now. It feels like much of this should be model-ed as part of the active flavor. But at this point its hard to see what the best way forward will be. Help with this effort would be very welcome: https://etherpad.openstack.org/p/YVR-nova-flavors-and-image-properties ... but I am still writing everything up from the summit, so we can get more folks to join in the action items from the summit sessions. More on that soon, I hope! Thanks, John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] correct API for getting image metadata for an instance ?
As part of the work to object-ify the image metadata dicts, I'm looking at the current way the libvirt driver fetches image metadata for an instance, in cases where the compute manager hasn't already passed it into the virt driver API. I see 2 methods that libvirt uses to get the image metadata - nova.utils.get_image_from_system_metadata(instance.system_metadata) It takes the system metadata stored against the instance and turns it into image metadata. - nova.compute.utils.get_image_metadata(context, image_api, instance.image_ref, instance) This tries to get metadata from the image api and turns this into system metadata It then gets system metadata from the instance and merges it from the data from the image It then calls nova.utils.get_image_from_system_metadata() IIUC, any changes against the image will override what is stored against the instance IIUC, when an instance is booted, the image metadata should be saved against the instance. So I'm wondering why we need to have code in compute.utils that merges back in the image metadata each time ? Is this intentional so that we pull in latest changes from the image, to override what's previously saved on the instance ? If so, then it seems that we should have been consistent in using the compute_utils get_image_metadata() API everywhere. It seems wrong though to pull in the latest metadata from the image. The libvirt driver makes various decisions at boot time about how to configure the guest based on the metadata. When we later do changes to that guest (snapshot, hotplug, etc, etc) we *must* use exactly the same image metadata we had at boot time, otherwise decisions we make will be inconsistent with how the guest is currently configured. eg if you set hw_disk_bus=virtio at boot time, and then later change the image to use hw_disk_bus=scsi, and then try to hotplug a new drive on the guest, we *must* operate wrt hw_disk_bus=virtio because the guest will not have any scsi bus present. I agree, as well, that we should use the system_metadate instead of getting the latest from Glance. I just wish there would be an easy way to edit it, in order to update some keys such as video driver, watchdog action, nic driver.. etc, so it would be picked up on hard reboot, for example. This says to me we should /never/ use the compute_utils get_image_metadata() API once the guest is running, and so we should convert libvirt to use nova.utils.get_image_from_system_metadata() exclusively. It also makes me wonder how nova/compute/manager.py is obtaining image meta in cases where it passes it into the API and whether that needs changing at all. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] correct API for getting image metadata for an instance ?
+1 But we should answer the following question before switching libvirt to utils.get_image_from_system_metadata. Some differences of utils.get_image_from_system_metadata from compute.utils.get_image_metadata are: 1) Instance metadata has a restriction of property length: long data is stripped to 255 chars. This may cause a fault on loading serialized structure from a metadata property. 2) utils.get_image_from_system_metadata skips properties defined by CONF.non_inheritable_image_properties So the question is: are these facts important for libvirt? On Thu, May 28, 2015 at 10:18 AM, Sahid Orentino Ferdjaoui sahid.ferdja...@redhat.com wrote: On Wed, May 27, 2015 at 02:59:10PM +0100, Daniel P. Berrange wrote: As part of the work to object-ify the image metadata dicts, I'm looking at the current way the libvirt driver fetches image metadata for an instance, in cases where the compute manager hasn't already passed it into the virt driver API. I see 2 methods that libvirt uses to get the image metadata - nova.utils.get_image_from_system_metadata(instance.system_metadata) It takes the system metadata stored against the instance and turns it into image metadata. - nova.compute.utils.get_image_metadata(context, image_api, instance.image_ref, instance) This tries to get metadata from the image api and turns this into system metadata It then gets system metadata from the instance and merges it from the data from the image It then calls nova.utils.get_image_from_system_metadata() IIUC, any changes against the image will override what is stored against the instance IIUC, when an instance is booted, the image metadata should be saved against the instance. So I'm wondering why we need to have code in compute.utils that merges back in the image metadata each time ? As you said during boot time we store in the instance system_metadata the image properties. Except for some special cases I don't see any reason to use the method 'get_image_metadata' and we should probably clean the code in libvirt. Is this intentional so that we pull in latest changes from the image, to override what's previously saved on the instance ? If so, then it seems that we should have been consistent in using the compute_utils get_image_metadata() API everywhere. It seems wrong though to pull in the latest metadata from the image. The libvirt driver makes various decisions at boot time about how to configure the guest based on the metadata. When we later do changes to that guest (snapshot, hotplug, etc, etc) we *must* use exactly the same image metadata we had at boot time, otherwise decisions we make will be inconsistent with how the guest is currently configured. eg if you set hw_disk_bus=virtio at boot time, and then later change the image to use hw_disk_bus=scsi, and then try to hotplug a new drive on the guest, we *must* operate wrt hw_disk_bus=virtio because the guest will not have any scsi bus present. This says to me we should /never/ use the compute_utils get_image_metadata() API once the guest is running, and so we should convert libvirt to use nova.utils.get_image_from_system_metadata() exclusively. It also makes me wonder how nova/compute/manager.py is obtaining image meta in cases where it passes it into the API and whether that needs changing at all. +1 Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] correct API for getting image metadata for an instance ?
On Wed, May 27, 2015 at 02:59:10PM +0100, Daniel P. Berrange wrote: As part of the work to object-ify the image metadata dicts, I'm looking at the current way the libvirt driver fetches image metadata for an instance, in cases where the compute manager hasn't already passed it into the virt driver API. I see 2 methods that libvirt uses to get the image metadata - nova.utils.get_image_from_system_metadata(instance.system_metadata) It takes the system metadata stored against the instance and turns it into image metadata. - nova.compute.utils.get_image_metadata(context, image_api, instance.image_ref, instance) This tries to get metadata from the image api and turns this into system metadata It then gets system metadata from the instance and merges it from the data from the image It then calls nova.utils.get_image_from_system_metadata() IIUC, any changes against the image will override what is stored against the instance IIUC, when an instance is booted, the image metadata should be saved against the instance. So I'm wondering why we need to have code in compute.utils that merges back in the image metadata each time ? As you said during boot time we store in the instance system_metadata the image properties. Except for some special cases I don't see any reason to use the method 'get_image_metadata' and we should probably clean the code in libvirt. Is this intentional so that we pull in latest changes from the image, to override what's previously saved on the instance ? If so, then it seems that we should have been consistent in using the compute_utils get_image_metadata() API everywhere. It seems wrong though to pull in the latest metadata from the image. The libvirt driver makes various decisions at boot time about how to configure the guest based on the metadata. When we later do changes to that guest (snapshot, hotplug, etc, etc) we *must* use exactly the same image metadata we had at boot time, otherwise decisions we make will be inconsistent with how the guest is currently configured. eg if you set hw_disk_bus=virtio at boot time, and then later change the image to use hw_disk_bus=scsi, and then try to hotplug a new drive on the guest, we *must* operate wrt hw_disk_bus=virtio because the guest will not have any scsi bus present. This says to me we should /never/ use the compute_utils get_image_metadata() API once the guest is running, and so we should convert libvirt to use nova.utils.get_image_from_system_metadata() exclusively. It also makes me wonder how nova/compute/manager.py is obtaining image meta in cases where it passes it into the API and whether that needs changing at all. +1 Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] correct API for getting image metadata for an instance ?
On Thu, May 28, 2015 at 11:55:59AM +0300, Feodor Tersin wrote: +1 But we should answer the following question before switching libvirt to utils.get_image_from_system_metadata. Some differences of utils.get_image_from_system_metadata from compute.utils.get_image_metadata are: 1) Instance metadata has a restriction of property length: long data is stripped to 255 chars. This may cause a fault on loading serialized structure from a metadata property. 2) utils.get_image_from_system_metadata skips properties defined by CONF.non_inheritable_image_properties So the question is: are these facts important for libvirt? compute.utils.get_image_metadata calls utils.get_image_from_system_metadata so those don't cause any difference in behaviour. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] correct API for getting image metadata for an instance ?
I mean the case: - an image has long property AAA and non inheritable property BBB; - an instance based on the image stores truncated AAA and full BBB; - utils.get_image_from_system_metadata returns truncated AAA but not BBB; - compute.utils.get_image_metadata updates the result of utils.get_image_from_system_metadata by the image metadata and returns full AAA and added BBB. This is the difference i mean. On Thu, May 28, 2015 at 11:59 AM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, May 28, 2015 at 11:55:59AM +0300, Feodor Tersin wrote: +1 But we should answer the following question before switching libvirt to utils.get_image_from_system_metadata. Some differences of utils.get_image_from_system_metadata from compute.utils.get_image_metadata are: 1) Instance metadata has a restriction of property length: long data is stripped to 255 chars. This may cause a fault on loading serialized structure from a metadata property. 2) utils.get_image_from_system_metadata skips properties defined by CONF.non_inheritable_image_properties So the question is: are these facts important for libvirt? compute.utils.get_image_metadata calls utils.get_image_from_system_metadata so those don't cause any difference in behaviour. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] correct API for getting image metadata for an instance ?
On Thu, May 28, 2015 at 12:22:59PM +0300, Feodor Tersin wrote: I mean the case: - an image has long property AAA and non inheritable property BBB; - an instance based on the image stores truncated AAA and full BBB; - utils.get_image_from_system_metadata returns truncated AAA but not BBB; - compute.utils.get_image_metadata updates the result of utils.get_image_from_system_metadata by the image metadata and returns full AAA and added BBB. This is the difference i mean. No, your last step is wrong. The code in compute_utils.get_image_metadata is this: # Get the system metadata from the instance system_meta = utils.instance_sys_meta(instance) # Merge the metadata from the instance with the image's, if any system_meta.update(image_system_meta) # Convert the system metadata to image metadata return utils.get_image_from_system_metadata(system_meta) So the image metadata and the instance system metadata are merged into the same dict, *before* the utils.get_image_from_system_metadata() method is invokved. So both sets of data will be truncated non-inheritable props dropped. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] correct API for getting image metadata for an instance ?
Yes, my mistake. I missed that utils.get_image_from_system_metadata just converts a passed metadata, but do not get metadata from the instance. So non-inhertable properties are omitted in both result sets. But compute.utils.get_image_metadata returns not trucated long properties. It reads truncated value by utils.instance_sys_meta and updates it to full value from image_system_metadata. So if replace call of compute.utils.get_image_metadata(.., instance) to utils.get_image_from_system_metadata(instance.system_metadata), it will get truncated props. On Thu, May 28, 2015 at 12:33 PM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, May 28, 2015 at 12:22:59PM +0300, Feodor Tersin wrote: I mean the case: - an image has long property AAA and non inheritable property BBB; - an instance based on the image stores truncated AAA and full BBB; - utils.get_image_from_system_metadata returns truncated AAA but not BBB; - compute.utils.get_image_metadata updates the result of utils.get_image_from_system_metadata by the image metadata and returns full AAA and added BBB. This is the difference i mean. No, your last step is wrong. The code in compute_utils.get_image_metadata is this: # Get the system metadata from the instance system_meta = utils.instance_sys_meta(instance) # Merge the metadata from the instance with the image's, if any system_meta.update(image_system_meta) # Convert the system metadata to image metadata return utils.get_image_from_system_metadata(system_meta) So the image metadata and the instance system metadata are merged into the same dict, *before* the utils.get_image_from_system_metadata() method is invokved. So both sets of data will be truncated non-inheritable props dropped. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
