[openstack-dev] [nova] Reconfiguring devices when boot from instance snapshot
There is a task - device reconfiguration when boot instance from instance snapshot. For example, one may want to change shutdown behaviour by nova boot ... --image --block_device device=/dev/vda,shutdown=false when originally shutdown attribute for /dev/vda is 'remove' in the instance snapshot. Now this feature is not implemented. In BlockDeviceConfig reference ( https://wiki.openstack.org/wiki/BlockDeviceConfig) the reconfiguration feature is mentioned very slightly (on 'no_device' attribute). Neigther this reference, nor current implementation of BDMv2 don't clear whether it's supposed to support the reconfiguration in future. Also it's not clear whether this feature can be implemented in current API. Is there any additional information about plans of using and improving BDMv2, particularly for the reconfiguration? Where can i see it? Moreover the future of 'image_ref' parameters is not clear. Although it's need to set it to boot instance from image, also image must be set in BDMv2. So this parameter seems partially obsolete. But how is planned to boot instance from instance snapshot with no using of this parameter? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Device names supplied to the boot request
> Date: Thu, 16 Jul 2015 20:10:45 +0100 > From: ndipa...@redhat.com > To: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [Nova] Device names supplied to the boot request > > On 07/16/2015 06:35 PM, Matt Riedemann wrote: > > > > > > On 7/16/2015 11:47 AM, Nikola Đipanov wrote: > >> On 07/16/2015 11:24 AM, Sean Dague wrote: > >>> On 07/15/2015 01:41 PM, Andrew Laski wrote: > On 07/15/15 at 12:19pm, Matt Riedemann wrote: > >>> > > The other part of the discussion is around the API changes, not just > > for libvirt, but having a microversion that removes the device from > > the request so it's no longer optional and doesn't provide some false > > sense that it works properly all of the time. We talked about this in > > the nova channel yesterday and I think the thinking was we wanted to > > get agreement on dropping that with a microversion before moving > > forward with the libvirt change you have to ignore the requested > > device name. > > > > From what I recall, this was supposed to really only work reliably > > for > > xen but now it actually might not, and would need to be tested again. > > Seems we could start by checking the xen CI to see if it is running > > the test_minimum_basic scenario test or anything in > > test_attach_volume.py in Tempest. > > This doesn't really work reliably for xen either, depending on what is > being done. For the xenapi driver Nova converts the device name > provided into an integer based on the trailing letter, so 'vde' becomes > 4, and asks xen to mount the device based on that int. Xen does honor > that integer request so you'll get an 'e' device, but you could be > asking for hde and get an xvde or vice versa. > >>> > >>> So this sounds like it's basically not working today. For Linux guests > >>> it really can't work without custom in guest code anyway, given how > >>> device enumeration works. > >>> > >>> That feels to me like we remove it from the API with a microversion, and > >>> when we do that just comment that trying to use this before that > >>> microversion is highly unreliable (possibly dangerous) and may just > >>> cause tears. > >>> > >> > >> Also, not being able to specify device names would make it impossible to > >> implement certain features that EC2 API can provide, such as overriding > >> the image block devices without significant effort. > > > > Huh? (x2) With your change you're ignoring the requested device name > > anyway, so how does this matter? Also, the ec2 API is moving out of > > tree so do we care what that means for the openstack compute API? > > > > Please look at the patch and the bug I link in the follow up email > (copied here for your convenience). It should be clearer then which > features cannot possibly work [1][2]. > > As for supporting the EC2 API - I don't know the answer to that if we > decide we don't care about them - that's cool with me. Even without that > as a consideration, I still think the current proposed patch is the best > way forward. > > [1] https://review.openstack.org/#/c/190324/ > [2] https://bugs.launchpad.net/nova/+bug/1370250 I confirm that your patch for bdm overloading does work correctly. I posted a link into your review on appropriate functional test [1] Nova EC2 is deprecated, but standalone ec2api is actual. Nova team promised: "Where needed, we will work this those projects to extending the Nova API such that its possible to add that functionality on top of the Nova API." [2] This (support of standalone ec2api) is the first reason to keep device names in API. The second reason is that the overloading feature is useful for native Nova users as well.Really if they have a standard image with multiple devices, the overloading feature provides a simpler way to boot an instance with small changes of standard devices. Than to create a new image with modified bdms, run the instance, and then manage the new image (keep it or remove). Or to boot from the image and then detach/create/attach devices (and this doesn't work for swap and ephemerals). Clearly AWS provides the simplest way to do this (adjusting, not only overloading). Perhaps sometime Nova will provide this too. As you mentioned above, device names are the only way to address to device for overloading. The third reason is that at least xen supports them.I know that vdr becomes /dev/xvdr, but:1) I believe the main aim of specifying device name is to easy distinguish among devices. A last char is enough for that. And xen supports that.2) Xen users may use this now.3) We don't know about device name support in VMWare and other hypervisors.4) AWS in its turn also does not warrant that selected name will be the same as requested one [3] So i think Nova could look at a specified device name as at a hint. If a used hypervisor can follow it even partially, Nova honours it.In particular libvirt could sort dev
Re: [openstack-dev] [nova][cinder][glance] Should we allow this kind of action?
+1 to the wonder We could have more explicit check in Cinder.Either check disk_format (it's None for volume-backed images)or add to Glance a new container_format type for this kind of images (it's also None currently) and check it. From: sxmatch1...@gmail.com Date: Mon, 27 Jul 2015 10:49:39 +0800 To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [nova][cinder] Should we allow this kind of action? I wonder why user need to use this kind of image(vm snapshot) to create volume, and then use this volume to create a vm again. It's meaningless I think. On the other hand, if an image size is zero, cinder shouldn't allow to create this volume from image. 2015-07-27 9:39 GMT+08:00 Zhenyu Zheng : Hi all, Recently, I've been asked to perform this kind of action using OpenStack: 1. Launch an volume-backended instance.2. Take a snapshot of this instance using nova image-create, an image will be added in glance, the size is zero, and the BDM will be saved in it's metadata.3. Create an volume using this image (with Cinder), This volume will be marked with bootable.4. Launch an new volume-backended instance using this newly built volume. There will be errors performing this action:1. Cinder will create an volume with zero size and the BDM saved in metadata is transformed from dict to string and it is not able to be used in nova for instance creation.2. The BDM is provided by user with REST API, and it will be conflict with the BDM saved in metadata. Now, my question is:Should we allow this kind of action in Nova? Or should we only use the image directly for instance creation. If this kind of action is not allowed, should we add checks in Cinder to forbid volume creation with an zero-sized image? Thanks, BR,Zhenyu __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best Wishes For You! __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] contextlib.nested and Python3 failing
> From: kevin.mitch...@rackspace.com > > Actually, there should no longer be a need to use contextlib.nested. > We've explicitly dropped Python 2.6 compatibility, which means we're > expecting compatibility with Python 2.7+ only, and as of Python 2.7, the > 'with' statement supports accepting multiple 'as' clauses. The > contextlib.nested tool was really only necessary to work around that > functionality being missing in Python 2.6, and has been deprecated as of > Python 2.7 because it's no longer necessary. 'with' statement has multiple form, but it does not allow line breaks inside. It leads to ugly formatting.See https://github.com/stackforge/ec2-api/commit/f5b320edbbc8ab554dae496dcae78be6fdd25fe7#diff-bfcbc316f2eb6b39de99a921a5753aadL631 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Fine-grained error reporting via the external API
> From: mbo...@redhat.com > To: openstack-dev@lists.openstack.org > Date: Fri, 11 Sep 2015 10:41:47 +0100 > Subject: [openstack-dev] [nova] Fine-grained error reporting via the > external API > > However, all I have is that I made a BadRequest. I could potentially > grep the human readable error message, but the text of that message > doesn't form part of the API, and it may be translated in any case. As > an API consumer, it seems I can't really tell anything other than 'it > didn't work'. More than that requires guesswork, heuristics and inference. > > I don't think I've missed some source of additional wisdom, but it would > obviously be great if I have. Has there ever been any effort to define > some contract around more fine-grained error reporting? Matt, iiuc this has been discussed early [1]. There is a review [2] mentioned at the end of that thread. [1] http://markmail.org/message/6l6szrm6ox7w2cxk [2] https://review.openstack.org/#/c/167793/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] About availability zones
Hi. Currently, when performing live-migration the AZ of the instance didn't update. In usecase like this:Instance_1 is in host1 which is in az1, we live-migrate it to host2 (provide host2 in API request) which is in az2. The operation will secusess but the availability zone data stored in instance1 is still az1, this may cause inconsistency with the az data stored in instance db and the actual az. I think update the az information in instance using the host az can solve this. Since a host can be included into different AZs, you probably need to allow a caller optionally to specify a target AZ. And as i understand the same problem occurs for resize, evacuate and other similar operations, doesn't it? __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] "correct" API for getting image metadata for an instance ?
+1 But we should answer the following question before switching libvirt to utils.get_image_from_system_metadata. Some differences of utils.get_image_from_system_metadata from compute.utils.get_image_metadata are: 1) Instance metadata has a restriction of property length: long data is stripped to 255 chars. This may cause a fault on loading serialized structure from a metadata property. 2) utils.get_image_from_system_metadata skips properties defined by CONF.non_inheritable_image_properties So the question is: are these facts important for libvirt? On Thu, May 28, 2015 at 10:18 AM, Sahid Orentino Ferdjaoui < sahid.ferdja...@redhat.com> wrote: > On Wed, May 27, 2015 at 02:59:10PM +0100, Daniel P. Berrange wrote: > > As part of the work to object-ify the image metadata dicts, I'm looking > > at the current way the libvirt driver fetches image metadata for an > > instance, in cases where the compute manager hasn't already passed it > > into the virt driver API. I see 2 methods that libvirt uses to get the > > image metadata > > > > - nova.utils.get_image_from_system_metadata(instance.system_metadata) > > > > It takes the system metadata stored against the instance > > and turns it into image metadata. > > > > > > - nova.compute.utils.get_image_metadata(context, > > image_api, > > instance.image_ref, > >instance) > > > > This tries to get metadata from the image api and turns > > this into system metadata > > > > It then gets system metadata from the instance and merges > > it from the data from the image > > > > It then calls nova.utils.get_image_from_system_metadata() > > > > IIUC, any changes against the image will override what > > is stored against the instance > > > > > > > > IIUC, when an instance is booted, the image metadata should be > > saved against the instance. So I'm wondering why we need to have > > code in compute.utils that merges back in the image metadata each > > time ? > > As you said during boot time we store in the instance system_metadata > the image properties. Except for some special cases I don't see any > reason to use the method 'get_image_metadata' and we should probably > clean the code in libvirt. > > > Is this intentional so that we pull in latest changes from the > > image, to override what's previously saved on the instance ? If > > so, then it seems that we should have been consistent in using > > the compute_utils get_image_metadata() API everywhere. > > > > It seems wrong though to pull in the latest metadata from the > > image. The libvirt driver makes various decisions at boot time > > about how to configure the guest based on the metadata. When we > > later do changes to that guest (snapshot, hotplug, etc, etc) > > we *must* use exactly the same image metadata we had at boot > > time, otherwise decisions we make will be inconsistent with how > > the guest is currently configured. > > > > eg if you set hw_disk_bus=virtio at boot time, and then later > > change the image to use hw_disk_bus=scsi, and then try to hotplug > > a new drive on the guest, we *must* operate wrt hw_disk_bus=virtio > > because the guest will not have any scsi bus present. > > > > This says to me we should /never/ use the compute_utils > > get_image_metadata() API once the guest is running, and so we > > should convert libvirt to use nova.utils.get_image_from_system_metadata() > > exclusively. > > > > It also makes me wonder how nova/compute/manager.py is obtaining image > > meta in cases where it passes it into the API and whether that needs > > changing at all. > > +1 > > > > > Regards, > > Daniel > > -- > > |: http://berrange.com -o- > http://www.flickr.com/photos/dberrange/ :| > > |: http://libvirt.org -o- > http://virt-manager.org :| > > |: http://autobuild.org -o- > http://search.cpan.org/~danberr/ :| > > |: http://entangle-photo.org -o- > http://live.gnome.org/gtk-vnc :| > > > > > __ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] "correct" API for getting image metadata for an instance ?
I mean the case: - an image has long property AAA and non inheritable property BBB; - an instance based on the image stores truncated AAA and full BBB; - utils.get_image_from_system_metadata returns truncated AAA but not BBB; - compute.utils.get_image_metadata updates the result of utils.get_image_from_system_metadata by the image metadata and returns full AAA and added BBB. This is the difference i mean. On Thu, May 28, 2015 at 11:59 AM, Daniel P. Berrange wrote: > On Thu, May 28, 2015 at 11:55:59AM +0300, Feodor Tersin wrote: > > +1 > > But we should answer the following question before switching libvirt to > > utils.get_image_from_system_metadata. > > > > Some differences of utils.get_image_from_system_metadata from > > compute.utils.get_image_metadata are: > > 1) Instance metadata has a restriction of property length: long data is > > stripped to 255 chars. > > This may cause a fault on loading serialized structure from a metadata > > property. > > > > 2) utils.get_image_from_system_metadata skips properties defined by > > CONF.non_inheritable_image_properties > > > > So the question is: are these facts important for libvirt? > > compute.utils.get_image_metadata calls utils.get_image_from_system_metadata > so those don't cause any difference in behaviour. > > Regards, > Daniel > -- > |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ > :| > |: http://libvirt.org -o- http://virt-manager.org > :| > |: http://autobuild.org -o- http://search.cpan.org/~danberr/ > :| > |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc > :| > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] "correct" API for getting image metadata for an instance ?
Yes, my mistake. I missed that utils.get_image_from_system_metadata just converts a passed metadata, but do not get metadata from the instance. So non-inhertable properties are omitted in both result sets. But compute.utils.get_image_metadata returns not trucated long properties. It reads truncated value by utils.instance_sys_meta and updates it to full value from image_system_metadata. So if replace call of compute.utils.get_image_metadata(.., instance) to utils.get_image_from_system_metadata(instance.system_metadata), it will get truncated props. On Thu, May 28, 2015 at 12:33 PM, Daniel P. Berrange wrote: > On Thu, May 28, 2015 at 12:22:59PM +0300, Feodor Tersin wrote: > > I mean the case: > > - an image has long property AAA and non inheritable property BBB; > > - an instance based on the image stores truncated AAA and full BBB; > > - utils.get_image_from_system_metadata returns truncated AAA but not BBB; > > - compute.utils.get_image_metadata updates the result of > > utils.get_image_from_system_metadata by the image metadata and returns > full > > AAA and added BBB. > > > > This is the difference i mean. > > No, your last step is wrong. > > The code in compute_utils.get_image_metadata is this: > > # Get the system metadata from the instance > system_meta = utils.instance_sys_meta(instance) > > # Merge the metadata from the instance with the image's, if any > system_meta.update(image_system_meta) > > # Convert the system metadata to image metadata > return utils.get_image_from_system_metadata(system_meta) > > So the image metadata and the instance system metadata are merged into the > same dict, *before* the utils.get_image_from_system_metadata() method is > invokved. So both sets of data will be truncated & non-inheritable props > dropped. > > Regards, > Daniel > -- > |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ > :| > |: http://libvirt.org -o- http://virt-manager.org > :| > |: http://autobuild.org -o- http://search.cpan.org/~danberr/ > :| > |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc > :| > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Availability of device names for operations with volumes and BDM and other features.
Nicola, i would add some words to Alexandre repsonse. We (standalone ec2api project guys) have filed some bugs (the main is [1]), but we don't know how to fix them since the way Nova's device names are moved on is unclear for us. Neither BP, nor wiki you've mentioned above don't explain what was happened with device names in images. Other bug which we filed by results of bdm v2 implementation [2] was resolved, but the fix returns only two devices (even if more than two volumes are defined in the image) instead of to write device names to the image and to return full bdm. I hope you will clarify this question (Alexandre referred to the patch with explicit elimination of device names for images). Also you mentioned that we can still use bdm v1. We do it now for instance launch, but we would like to switch to v2 to use new features like blank volumes which are provided by AWS as well. However v2 based launch has a suspicious feature which i asked about in ML [3], but no one answered me. It would be great if you clarify that question too. [1] https://bugs.launchpad.net/nova/+bug/1370177 [2] https://bugs.launchpad.net/nova/+bug/1370265 [3] http://lists.openstack.org/pipermail/openstack-dev/2015-May/063769.html On Wed, May 27, 2015 at 8:55 PM, Nikola Đipanov wrote: > On 05/27/2015 09:47 AM, Alexandre Levine wrote: > > Hi all, > > > > I'd like to bring up this matter again, although it was at some extent > > discussed during the recent summit. > > > > The problem arises from the fact that the functionality exposing device > > names for usage through public APIs is deteriorating in nova. It's being > > deliberately removed because as I understand, it doesn't universally and > > consistently work in all of the backends. It happens since IceHouse and > > introduction of bdm v2. The following very recent review is one of the > > ongoing efforts in this direction: > > https://review.openstack.org/#/c/185438/ > > > > I've abandoned the change as it is clear we need to discuss how to go > about this some more. > > But first let me try to give a bit more detailed explanation and > background to what the deal is with device names. Supplying device names > that will be honoured by the guests is really only possible with Xen PV > guests (meaning the guest needs to be running PV-enabled kernel and > drivers). > > Back in Havana, when we were working on [1] (see [2] for more details) > the basic idea was that we will still accept device names because > removing them from the public API is not likely to happen (mostly > because of the EC2 compatibility), but in case of libvirt driver, we > will treat them as hints only, and provide our own (by mostly > replicating the logic libvirt uses to order devices [3]). We also > allowed for device names to not be specified by the user as this is > really what anyone not using the EC2 API should be doing (users using > the EC2 API do however need to be aware the fact that i may not be > honoured). > > [1] > https://blueprints.launchpad.net/nova/+spec/improve-block-device-handling > [2] https://wiki.openstack.org/wiki/BlockDeviceConfig > [3] > > https://github.com/openstack/nova/blob/master/nova/virt/libvirt/blockinfo.py > > > The reason for my concern is that EC2 API have some important cases > > relying on this information (some of them have no workarounds). Namely: > > 1. Change of parameters set by image for instance booting. > > 2. Showing instance's devices information by euca2ools. > > 3. Providing additional volumes for instance booting > > 4. Attaching volume > > etc... > > > > So based on the above - it seems to me that you think we are removing > the information about device names completely. That's not the case - > currently it is simply not mandatory for the Nova boot API call (it was > never mandatory for volume attach afaict) - you can still pass it in, > though libvirt may not honour it. It will still be tracked by the Nova > DB and available for users to refer to. > > > Related to device names and additional related features we have troubles > > with now: > > 1. All device name related features > > As I said - they are not removed, in addition, you can still completely > disregard the BDMv2 syntax as Nova should transparently handle old-style > syntax when passed in (actually since BDM info is stored with images > when snapshotting and it may have been v1 syntax - it is likely that we > will never remove this support). If you are seeing some bugs related to > this - please report them. > > > 2. Modification of deleteOnTermination flag > > I don't have enough details on this but if some behaviour has changed > when using the old syntax - it is likely a bug so please report it. > > > 3. Modification of parameters for instance booting > > Again - I am not sure what this is related to exactly - but none of the > parameters have changed really (only new ones were added). It would be > good to get more information on this (preferably a bug report). > > > 4. deleteOnTer
[openstack-dev] [Nova] Let's set instance.root_gb=0 for volume-backed instances
There are several problems in algorithms which calculate size of host local disk space consumed by a volume-backed instance [1]. Obviously the volume-backed instance does not use the host local disk for the root volume. But these algorithms use root_gb, min_size, size which are not 0 for volume-backed instance. This leads to the problems. Some solutions have been proposed to solve these problems [2]. One problem was even discussed here [3]. However all these solutions are aimed at correcting specific algorithms to allow them to take into account a type of an instance. But this does not affect other algorithms, which also use root_gb as local space size [4]. I.e. other similar problems are retained. I propose to fix all of this by setting 0 for root_gb, min_size, size [5], since 0 is a natural value of these attributes. Any objections? [1] https://bugs.launchpad.net/nova/+bug/1334974 https://bugs.launchpad.net/nova/+bug/1459491 https://bugs.launchpad.net/nova/+bug/1466305 https://bugs.launchpad.net/nova/+bug/1457517 [2] https://review.openstack.org/#/c/136284/ https://review.openstack.org/#/c/184982/ https://review.openstack.org/#/c/186247/ https://review.openstack.org/#/c/192971/ [3] http://lists.openstack.org/pipermail/openstack-dev/2015-March/058138.html [4] https://github.com/openstack/nova/blob/master/nova/notifications.py#L407https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L250 [5] https://bugs.launchpad.net/nova/+bug/1469179 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api][nova] Openstack HTTP error codes
Hi Ken, 1 imageRef isn't the only attribute, which could receive an image id. There are kernelId, ramdiskId, and even bdm v2 as well. So we couldn't guess which attribute has the invalid value. 2 Besides NotFound case, other mixed cases are there. Such as attaching of a volume. A mountpoint can be busy, or the volume can be used by an other instance - both cases generate a conflict error. Do you suggest to use specially formatted message in all such cases (when the same http error code has several reasons)? But to make a work with Nova API straightforward, all messages should have this format, even in simplest cases. 3 How to parse a localized message? A Nova API client shouldn't use en_us locale only to communicate with Nova, because it should display messages generated by Nova to an end user. On Mon, Feb 2, 2015 at 8:28 AM, Ken'ichi Ohmichi wrote: > 2015-01-30 18:13 GMT+09:00 Simon Pasquier : > > On Fri, Jan 30, 2015 at 3:05 AM, Kenichi Oomichi < > oomi...@mxs.nes.nec.co.jp> > > wrote: > >> > >> > -Original Message- > >> > From: Roman Podoliaka [mailto:rpodoly...@mirantis.com] > >> > Sent: Friday, January 30, 2015 2:12 AM > >> > To: OpenStack Development Mailing List (not for usage questions) > >> > Subject: Re: [openstack-dev] [api][nova] Openstack HTTP error codes > >> > > >> > Hi Anne, > >> > > >> > I think Eugeniya refers to a problem, that we can't really distinguish > >> > between two different badRequest (400) errors (e.g. wrong security > >> > group name vs wrong key pair name when starting an instance), unless > >> > we parse the error description, which might be error prone. > >> > >> Yeah, current Nova v2 API (not v2.1 API) returns inconsistent messages > >> in badRequest responses, because these messages are implemented at many > >> places. But Nova v2.1 API can return consistent messages in most cases > >> because its input validation framework generates messages > >> automatically[1]. > > > > > > When you say "most cases", you mean JSON schema validation only, right? > > IIUC, this won't apply to the errors described by the OP such as invalid > key > > name, unknown security group, ... > > Yeah, right. > I implied that in "most cases" and I have one patch[1] for covering them. > By the patch, the sample messages also will be based on the same > format and be consistent. > The other choice we have is CamelCase exception as the fist mail, that > also is interesting. > > Thanks > Ken Ohmichi > > --- > [1]: https://review.openstack.org/#/c/151954 > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [NOVA] security group fails to attach to an instance if port-id is specified during boot.
nova boot ... --nic port-id=xxx --nic net-id=yyy this case is valid, right? I.e. i want to boot instance with two ports. The first port is specified, but the second one is created at network mapping stage. If i specify a security group as well, it will be used for the second port (if not - default group will): nova boot ... --nic port-id=xxx --nic net-id=yyy --security-groups sg-1 Thus a port and a security group can be specified together. On Mon, Feb 9, 2015 at 7:14 PM, Matt Riedemann wrote: > > > On 9/26/2014 3:19 AM, Christopher Yeoh wrote: > >> On Fri, 26 Sep 2014 11:25:49 +0400 >> Oleg Bondarev wrote: >> >> On Fri, Sep 26, 2014 at 3:30 AM, Day, Phil wrote: >>> >>>I think the expectation is that if a user is already interaction with Neutron to create ports then they should do the security group assignment in Neutron as well. >>> Agree. However what do you think a user expects when he/she boots a >>> vm (no matter providing port_id or just net_id) >>> and specifies security_groups? I think the expectation should be that >>> instance will become a member of the specified groups. >>> Ignoring security_groups parameter in case port is provided (as it is >>> now) seems completely unfair to me. >>> >> >> One option would be to return a 400 if both port id and security_groups >> is supplied. >> >> Chris >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > Coming back to this, we now have a change from Oleg [1] after an initial > attempt that was reverted because it would break server creates if you > specified a port (because the original change would blow up when the > compute API added the 'default' security group to the request'). > > The new change doesn't add the 'default' security group to the request so > if you specify a security group and port on the request, you'll now get a > 400 error response. > > Does this break API compatibility? It seems this falls under the first > bullet here [2], "A change such that a request which was successful before > now results in an error response (unless the success reported previously > was hiding an existing error condition)". Does that caveat in parenthesis > make this OK? > > It seems like we've had a lot of talk about warts in the compute v2 API > for cases where an operation is successful but didn't yield the expected > result, but we can't change them because of API backwards compatibility > concerns so I'm hesitant on this. > > We also definitely need a Tempest test here, which I'm looking into. I > think I can work this into the test_network_basic_ops scenario test. > > [1] https://review.openstack.org/#/c/154068/ > [2] https://wiki.openstack.org/wiki/APIChangeGuidelines# > Generally_Not_Acceptable > > -- > > Thanks, > > Matt Riedemann > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][neutron] Create VM using port-create vs nova boot only?
Hi When you create a port separately, you can specify additional fixed IPs, extra DHCP options. But with 'nova boot' you cannot. Also if you need an instance with several nics, and you want that each nic has its own set of security groups, you should create ports separately. Because 'nova boot --security-groups ggg' command sets specified security groups for an each port, which is created during the instance launch. On Tue, Feb 10, 2015 at 9:21 AM, Wanjing Xu wrote: > There seemed to be two ways to create a VM via cli: > > 1) use neutron command to create a port first and then use nova command to > attach the vm to that port(neutron port-create.. followed by nova boot > --nic port-id=) > 2)Just use nova command and a port will implicitly be created for you(nova > boot --nic net-id=net-uuid). > > My question is : is #2 sufficient enough to cover all the scenarios? In > other words, if we are not allowed to use #1(can only use #2 to create vm), > would we miss anything? > > Regards! > Wanjing Xu > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [NOVA] security group fails to attach to an instance if port-id is specified during boot.
I definitely don't expect any change of the existing port in the case with two nics. However in the case of single nic a question like 'what is impact of security-groups parameter' arises. Also a similar question arises out of '--nic port-id=xxx,v4-fixed-ip=yyy' combination. Moreover, if we assume that, for example, security-groups parameter affects the specified port, the next question is 'what is the result group set'. Does it replace groups of the port, or just update them? Thus i agree with you, that this part of nova API is not clear now. But the case with two nics make sense, works now, and can be used by someone. Do you really want to break it? On Tue, Feb 10, 2015 at 10:29 AM, Oleg Bondarev wrote: > > > On Mon, Feb 9, 2015 at 8:50 PM, Feodor Tersin > wrote: > >> nova boot ... --nic port-id=xxx --nic net-id=yyy >> this case is valid, right? >> I.e. i want to boot instance with two ports. The first port is specified, >> but the second one is created at network mapping stage. >> If i specify a security group as well, it will be used for the second >> port (if not - default group will): >> nova boot ... --nic port-id=xxx --nic net-id=yyy --security-groups sg-1 >> Thus a port and a security group can be specified together. >> > > The question here is what do you expect for the existing port - it's > security groups updated or not? > Will it be ok to silently (or with warning in logs) ignore security groups > for it? > If it's ok then is it ok to do the same for: > nova boot ... --nic port-id=xxx --security-groups sg-1 > where the intention is clear enough? > > >> >> On Mon, Feb 9, 2015 at 7:14 PM, Matt Riedemann < >> mrie...@linux.vnet.ibm.com> wrote: >> >>> >>> >>> On 9/26/2014 3:19 AM, Christopher Yeoh wrote: >>> >>>> On Fri, 26 Sep 2014 11:25:49 +0400 >>>> Oleg Bondarev wrote: >>>> >>>> On Fri, Sep 26, 2014 at 3:30 AM, Day, Phil wrote: >>>>> >>>>>I think the expectation is that if a user is already interaction >>>>>> with Neutron to create ports then they should do the security group >>>>>> assignment in Neutron as well. >>>>>> >>>>>> >>>>> Agree. However what do you think a user expects when he/she boots a >>>>> vm (no matter providing port_id or just net_id) >>>>> and specifies security_groups? I think the expectation should be that >>>>> instance will become a member of the specified groups. >>>>> Ignoring security_groups parameter in case port is provided (as it is >>>>> now) seems completely unfair to me. >>>>> >>>> >>>> One option would be to return a 400 if both port id and security_groups >>>> is supplied. >>>> >>>> Chris >>>> >>>> ___ >>>> OpenStack-dev mailing list >>>> OpenStack-dev@lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>> Coming back to this, we now have a change from Oleg [1] after an initial >>> attempt that was reverted because it would break server creates if you >>> specified a port (because the original change would blow up when the >>> compute API added the 'default' security group to the request'). >>> >>> The new change doesn't add the 'default' security group to the request >>> so if you specify a security group and port on the request, you'll now get >>> a 400 error response. >>> >>> Does this break API compatibility? It seems this falls under the first >>> bullet here [2], "A change such that a request which was successful before >>> now results in an error response (unless the success reported previously >>> was hiding an existing error condition)". Does that caveat in parenthesis >>> make this OK? >>> >>> It seems like we've had a lot of talk about warts in the compute v2 API >>> for cases where an operation is successful but didn't yield the expected >>> result, but we can't change them because of API backwards compatibility >>> concerns so I'm hesitant on this. >>> >>> We also definitely need a Tempest test here, which I'm looking into. I >>> think I can work this into the test_network_basic_ops scenario test. >>> >>> [1] https://review.openstack.org/#/c/154068/ >
Re: [openstack-dev] [NOVA] security group fails to attach to an instance if port-id is specified during boot.
You could rise an exception if ports are specified for all nics. [1] I'm not sure that logging of this case is helpful, because only admins can access to logs. Probably the better way to warn a user is to do it at client side by nova cli (i.e. no any modification of nova server is needed). [1] It returns us back to the original Matt's question. I suppose that most people, which tried to specify security groups with ports, already found that it doesn't do work properly. And now they don't use them together. Other part, which didn't found this, have the error in their scripts (because they start instances with no expected groups). So rising an error in this case could be useful. In my turn, i used these parameters together ( https://github.com/stackforge/ec2-api/blob/master/ec2api/api/instance.py#L770) to do a workaround of a strange bug ( https://bugs.launchpad.net/nova/+bug/1384347), which is not actual since Juno. I believe OpenStack could not support compatibility for such workarounds. Finlally, the only my concern is the case with two nics, mentioned at the beginning. On Wed, Feb 11, 2015 at 10:39 AM, Oleg Bondarev wrote: > > > On Tue, Feb 10, 2015 at 5:26 PM, Feodor Tersin > wrote: > >> I definitely don't expect any change of the existing port in the case >> with two nics. However in the case of single nic a question like 'what is >> impact of security-groups parameter' arises. >> Also a similar question arises out of '--nic port-id=xxx,v4-fixed-ip=yyy' >> combination. >> Moreover, if we assume that, for example, security-groups parameter >> affects the specified port, the next question is 'what is the result group >> set'. Does it replace groups of the port, or just update them? >> >> Thus i agree with you, that this part of nova API is not clear now. But >> the case with two nics make sense, works now, and can be used by someone. >> Do you really want to break it? >> > > I don't want to break anything :) > I guess the only option then is to just log a warning that security groups > are ignored in case port_id is provided on boot - > but this still leaves a chance of broken user expectations. > > >> >> >> On Tue, Feb 10, 2015 at 10:29 AM, Oleg Bondarev >> wrote: >> >>> >>> >>> On Mon, Feb 9, 2015 at 8:50 PM, Feodor Tersin >>> wrote: >>> >>>> nova boot ... --nic port-id=xxx --nic net-id=yyy >>>> this case is valid, right? >>>> I.e. i want to boot instance with two ports. The first port is >>>> specified, but the second one is created at network mapping stage. >>>> If i specify a security group as well, it will be used for the second >>>> port (if not - default group will): >>>> nova boot ... --nic port-id=xxx --nic net-id=yyy --security-groups sg-1 >>>> Thus a port and a security group can be specified together. >>>> >>> >>> The question here is what do you expect for the existing port - it's >>> security groups updated or not? >>> Will it be ok to silently (or with warning in logs) ignore security >>> groups for it? >>> If it's ok then is it ok to do the same for: >>> nova boot ... --nic port-id=xxx --security-groups sg-1 >>> where the intention is clear enough? >>> >>> >>>> >>>> On Mon, Feb 9, 2015 at 7:14 PM, Matt Riedemann < >>>> mrie...@linux.vnet.ibm.com> wrote: >>>> >>>>> >>>>> >>>>> On 9/26/2014 3:19 AM, Christopher Yeoh wrote: >>>>> >>>>>> On Fri, 26 Sep 2014 11:25:49 +0400 >>>>>> Oleg Bondarev wrote: >>>>>> >>>>>> On Fri, Sep 26, 2014 at 3:30 AM, Day, Phil >>>>>>> wrote: >>>>>>> >>>>>>>I think the expectation is that if a user is already interaction >>>>>>>> with Neutron to create ports then they should do the security group >>>>>>>> assignment in Neutron as well. >>>>>>>> >>>>>>>> >>>>>>> Agree. However what do you think a user expects when he/she boots a >>>>>>> vm (no matter providing port_id or just net_id) >>>>>>> and specifies security_groups? I think the expectation should be that >>>>>>> instance will become a member of the specified groups. >>>>>>> Ignoring security_groups parameter in case port is provided (as it is >>>>>>> now) seems completely u
Re: [openstack-dev] [nova] what's the merge plan for current proposed microversions?
Since the first microversion is merged ( https://review.openstack.org/#/c/140313/), please, pay attention to a novaclient upgrade to work with microversions ( https://review.openstack.org/#/c/152569/) On Mon, Mar 2, 2015 at 2:30 PM, Sean Dague wrote: > This change for the additional attributes for ec2 looks like it's > basically ready to go, except it has the wrong microversion on it (as > they anticipated the other changes landing ahead of them) - > https://review.openstack.org/#/c/155853 > > What's the plan for merging the outstanding microversions? I believe > we're all conceptually approved on all them, and it's an important part > of actually moving forward on the new API. It seems like we're in a bit > of a holding pattern on all of them right now, and I'd like to make sure > we start merging them this week so that they have breathing space before > the freeze. > > -Sean > > -- > Sean Dague > http://dague.net > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][ec2-api] Need advice about running Tempest against stackforge/ec2-api
Hi Sean On Mon, Apr 6, 2015 at 7:34 PM, Sean Dague wrote: > On 04/06/2015 12:13 PM, Andrey M. Pavlov wrote: > > Hi, > > > > We've got a couple of problems running original Tempest EC2 API test > against new standalone stackforge/ec2-api project and > > I wanted to ask for some advice about how to deal with it. > > > > Tempest now is running against our ec2-api after this review was closed - > > https://review.openstack.org/#/c/170258/ > > > > And now we face two problems (that can also be found in tempest logs of > this review - > > https://review.openstack.org/#/c/170668/) > > For now I switched tempest gating job to non-voting until these problems > are resolved in the following review - > > https://review.openstack.org/#/c/170646/ > > > > Problems are: > > 1) > tempest.thirdparty.boto.test_ec2_network.EC2NetworkTest.test_disassociate_not_associated_floating_ip > > this test tries to allocate address and disassociate it without > association. > > Amazon allows to do it and does not throw error. But EC2 implementation > in Nova throws error. > > We have the same test in our own test suite against stackforge/ec2-api > (but it's not merged yet) and I checked it against Amazon. > > I suggest to remove this test from tempest as incompatible with Amazon. > > Also it can be skipped but for me it has no sense. > > This seems fine as a removal. > > > 2) > tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_compute_with_volumes > > This test registers three images by their manifests, run instance with > image/kernel/ramdisk parameters, > > and ssh into this instance to check something. > > This is not the only test that runs instance with such parameters but > this is the only one > > that ssh-s into such an instance. > > This instance runs but test can't ssh into it and it fails. Because this > instance doesn't have ramdisk and kernel. > > It runs supplied with image property only. The VM comes up > semi-functional and instance can't boot up as a result. > > Problem is in the ec2-api/nova communication. Nova public API doesn't > support kernel and ramdisk parameters during instance creation. > > > > Next I'll file a bug to ec2-api with this description. > > This seems problematic, because I think what you are saying is that the > stackforge EC2 API can't start a working guest. This is the only one of > the ec2 tests that actually validates the guest is running correctly IIRC. > > Is there an equivalent test that exists that you think would be better? > I'm also not sure I understand where the breakdown is here in missing > functionality. > I suggest to fix the test to fit both Nova EC2 and ec2api restrictions. Ec2api ignores ari/aki parameters for RunInstances operation, but supports registration of an ami image linked to ari and aki ones. Nova EC2 ignores the links in image registrations, but supports ari/aki parameters for RunInstances operation. So we could set these parameters for both operations to pass this test agains both Nova EC2 and ec2api. I've propesed a change for this: https://review.openstack.org/#/c/171222/ > > In the long run we should discuss adding this feature to public API but > for now we'd like to put Tempest > > in our project back to voting state. > > We've got several options about what to do for this and we need some > help to pick one (or several): > > 1) skip this test in tempest and switch tempest back to voting state in > our project. > > The problem is that this test is still also employed against nova's EC2 > so it'll get skipped there as well. > > 2) Leave it as non-voting until extension is added to nova. > > Great solution but it'll take way too long I understand. > > 3) add special condition to skipping the test so that it's skipped only > when employed against stackforge/ec2-api, > > while still working against nova if it's possible at all and not too > much hassle. > > > > Kind regards, > > Andrey. > > > > > __ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > -- > Sean Dague > http://dague.net > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][ec2-api] Need advice about running Tempest against stackforge/ec2-api
Hi. As you can see adjusted Tempest (https://review.openstack.org/#/c/171222/) runs well against both Nova EC2 and ec2api ( https://review.openstack.org/#/c/172059). On Tue, Apr 7, 2015 at 5:50 PM, Feodor Tersin wrote: > Hi Sean > > > On Mon, Apr 6, 2015 at 7:34 PM, Sean Dague wrote: > >> On 04/06/2015 12:13 PM, Andrey M. Pavlov wrote: >> > Hi, >> > >> > We've got a couple of problems running original Tempest EC2 API test >> against new standalone stackforge/ec2-api project and >> > I wanted to ask for some advice about how to deal with it. >> > >> > Tempest now is running against our ec2-api after this review was closed >> - >> > https://review.openstack.org/#/c/170258/ >> > >> > And now we face two problems (that can also be found in tempest logs of >> this review - >> > https://review.openstack.org/#/c/170668/) >> > For now I switched tempest gating job to non-voting until these >> problems are resolved in the following review - >> > https://review.openstack.org/#/c/170646/ >> > >> > Problems are: >> > 1) >> tempest.thirdparty.boto.test_ec2_network.EC2NetworkTest.test_disassociate_not_associated_floating_ip >> > this test tries to allocate address and disassociate it without >> association. >> > Amazon allows to do it and does not throw error. But EC2 implementation >> in Nova throws error. >> > We have the same test in our own test suite against stackforge/ec2-api >> (but it's not merged yet) and I checked it against Amazon. >> > I suggest to remove this test from tempest as incompatible with Amazon. >> > Also it can be skipped but for me it has no sense. >> >> This seems fine as a removal. >> >> > 2) >> tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_compute_with_volumes >> > This test registers three images by their manifests, run instance with >> image/kernel/ramdisk parameters, >> > and ssh into this instance to check something. >> > This is not the only test that runs instance with such parameters but >> this is the only one >> > that ssh-s into such an instance. >> > This instance runs but test can't ssh into it and it fails. Because >> this instance doesn't have ramdisk and kernel. >> > It runs supplied with image property only. The VM comes up >> semi-functional and instance can't boot up as a result. >> > Problem is in the ec2-api/nova communication. Nova public API doesn't >> support kernel and ramdisk parameters during instance creation. >> > >> > Next I'll file a bug to ec2-api with this description. >> >> This seems problematic, because I think what you are saying is that the >> stackforge EC2 API can't start a working guest. This is the only one of >> the ec2 tests that actually validates the guest is running correctly IIRC. >> >> Is there an equivalent test that exists that you think would be better? >> I'm also not sure I understand where the breakdown is here in missing >> functionality. >> > > I suggest to fix the test to fit both Nova EC2 and ec2api restrictions. > Ec2api ignores ari/aki parameters for RunInstances operation, but supports > registration of an ami image linked to ari and aki ones. > Nova EC2 ignores the links in image registrations, but supports ari/aki > parameters for RunInstances operation. > So we could set these parameters for both operations to pass this test > agains both Nova EC2 and ec2api. > > I've propesed a change for this: https://review.openstack.org/#/c/171222/ > > > >> > In the long run we should discuss adding this feature to public API but >> for now we'd like to put Tempest >> > in our project back to voting state. >> > We've got several options about what to do for this and we need some >> help to pick one (or several): >> > 1) skip this test in tempest and switch tempest back to voting state in >> our project. >> > The problem is that this test is still also employed against nova's EC2 >> so it'll get skipped there as well. >> > 2) Leave it as non-voting until extension is added to nova. >> > Great solution but it'll take way too long I understand. >> > 3) add special condition to skipping the test so that it's skipped only >> when employed against stackforge/ec2-api, >> > while still working against nova if it's possible at all and not too >> much hassle. &g
Re: [openstack-dev] [infra][glanceclient][cinderclient] Problems with juno check jobs
I proposed https://review.openstack.org/#/c/172522/ to fix this for all projects whose versions are restricted by global requirements. On Mon, Apr 13, 2015 at 5:55 PM, wrote: > Hi Gorka, > > Glance is seeing something very similar [3]. > > I've updated the two bugs ([1],[3]) with some extra info. > Both issues seem to have started around April 7th. > > Would anyone from infra be able to take a quick look? > > Thanks, > > -Stuart > > [3] https://bugs.launchpad.net/glance/+bug/1442682 > > Date: Mon, 13 Apr 2015 13:35:42 +0200 >> From: Gorka Eguileor >> To: openstack-dev@lists.openstack.org >> Subject: [openstack-dev] [Heat] [Cinderclient] All patches getting -1 >> on Cinderclient >> Message-ID: <20150413113541.ga23...@mail.corp.redhat.com> >> Content-Type: text/plain; charset=us-ascii >> >> Hi all, >> >> Currently all patches in Cinderclient are getting -1 from Jenkins >> because gate-tempest-dsvm-neutron-src-python-cinderclient-juno is >> failing. >> >> I opened a LP bug [1] on this, but basically the issue comes from Heat's >> requirements cap on Cinderclient [2] to an upper bound of 1.1.1 when >> current version is reported as 1.1.1.post100. >> >> So if you're getting -1 from that job, remember it's not you. ;-) >> >> >> Cheers, >> Gorka. >> >> [1] https://bugs.launchpad.net/tempest/+bug/1442086 >> [2] https://github.com/openstack/heat/blob/stable/juno/ >> requirements.txt#L25 >> > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] novaclient 'stable-compat-jobs-{name}' broken
I proposed a solution (https://review.openstack.org/#/c/172522/)
But i have no idea how to push it.
btw. As i see, glanceclient developers decided to just remove these jobs (
https://review.openstack.org/#/c/172999/)
On Tue, Apr 14, 2015 at 11:47 AM, Andrey Kurilin
wrote:
> >Given that, is it okay if I propose a patch to remove the
> 'stable-compat-jobs-{name}' build jobs for python-novaclient in
> project-config?
>
> I suppose we should remove 'stable-compat-jobs-{name}', since novaclient
> is already blocked for one week and it looks like there are no another
> solutions for now.
>
> On Fri, Apr 10, 2015 at 8:02 AM, melanie witt wrote:
>
>> Hi all,
>>
>> The following 'stable-compat-jobs-{name}' build jobs have been broken the
>> past two days, blocking all novaclient patches from passing jenkins checks:
>>
>> gate-tempest-dsvm-neutron-src-python-novaclient-icehouse
>> gate-tempest-dsvm-neutron-src-python-novaclient-juno
>>
>> The original purpose of these jobs was to check that patches proposed to
>> master wouldn't break in stable branches. This was before we started
>> pinning novaclient versions on stable branches. Now that we've pinned, the
>> way these jobs were passing was by installing the current novaclient, then
>> uninstalling it, then installing a version from pypi that fits within the
>> global requirements for the branch in question (icehouse or juno), then
>> running the tests.
>>
>> Well, recently this stopped working because for some reason, devstack is
>> no longer able to uninstall the latest version:
>>
>> Found existing installation: python-novaclient 2.23.0.post14
>> Can't uninstall 'python-novaclient'. No files were found to uninstall.
>>
>> And then I see the following error as a result:
>>
>> pkg_resources.ContextualVersionConflict: (python-novaclient 2.23.0.post14
>> (/opt/stack/new/python-novaclient),
>> Requirement.parse('python-novaclient<=2.20.0,>=2.18.0'),
>> set(['ceilometer']))
>>
>> I asked about this in #openstack-infra and was told we really shouldn't
>> be running the "src" build jobs on patches proposed to master anyhow, and
>> that it's not the right flow for devstack to install the latest, then
>> uninstall it, then install an older global reqs compatible version anyway.
>>
>> Given that, is it okay if I propose a patch to remove the
>> 'stable-compat-jobs-{name}' build jobs for python-novaclient in
>> project-config?
>>
>> Then after that, how are we supposed to go about cutting stable branches
>> for novaclient? And how can we get the 'stable-compat-jobs-{name}' jobs
>> running on only those respective branches? In project-config I didn't
>> understand how to limit build jobs to only patches proposed to a stable
>> branch.
>>
>> I'd appreciate any insights.
>>
>> Thanks,
>> melanie (melwitt)
>>
>>
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Best regards,
> Andrey Kurilin.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] ImageRef vs bdm v2 in REST API
Hi.
Since that bdm v2 was introduced for Havana, it requires a caller to
specify bdm for an image together with imageRef to boot an instance in a
case of using bdm v2 to attach additional volumes.
{"server": {"imageRef": "xxx",
"block_device_mapping_v2": [
{"uuid": "xxx",
"source_type": "image",
"destination_type": "local",
"boot_index": 0,
},
],
...}}
If we specify imageRef or the bdm record only, the launch will be failed.
Novaclient does it [1] for shell invokes like
nova boot --image xxx --block_device ...
, but does nothing for client API calls.
Such feature of usage is unclear. I've not found any documentation for the
usage. Wiki [2] doesn't mention this feature, but refers to a deleted API
sample [3]. Before deleting [4] that sample didn't consider the feature, so
it was wrong.
As a result the need to specify an image in two arguments seems a temprary
workaround for some problems. And whole bdm v2 conception looks not well
designed, not finished.
The question is: is this feature correct? Shall we specify an image twice
for the long term? Otherwise which manner should be established finally: by
imageRef or by an bdm record?
ps. There are a review [5] and a linked bug [6] which require clarification
of this question.
[1]
https://github.com/openstack/python-novaclient/commit/6a85c954c53f868251413db51cc1d9616acd4d02#diff-4812fe2b8b37d18cf9498f9fbbab17beR125
[2]
https://wiki.openstack.org/wiki/BlockDeviceConfig#API_data_model_and_backwards_compat_issues
[3]
https://github.com/openstack/nova/tree/master/doc/api_samples/os-block-device-mapping-v2-boot
[4]
https://github.com/openstack/nova/blob/2f32996c3e5625245a4d0588ab32880d41400b9e/doc/api_samples/os-block-device-mapping-v2-boot/server-post-req.json
[5] https://review.openstack.org/#/c/171984/
[6] https://bugs.launchpad.net/nova/+bug/1441990
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] The security group is confused in the create server api
> > > nova boot test --flavor m1.tiny --security_group ssh --security_group > http --nic net-id=xxx --nic port-id=yyy > > Which port/nic the security group should bind to? the first one? both? > Both groups will be set to a new port of xxx network. A set of port yyy SG will not be changed. This question was discussed here recently: http://lists.openstack.org/pipermail/openstack-dev/2015-February/056339.html. See also https://review.openstack.org/#/c/154068/ with its comments. > > The legacy network will be removed and I think it is the time to change > the API to satisfied the new situation. May be the new cli like below: > > nova boot test --flavor m1.tiny --nic > net-id=xx,security_group=ssh --nic port-id=y --nic > net-id=,security_group=http,security_group=mysql > I agree, API have to be more straightforward. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] The security group is confused in the create server api
On Wed, May 13, 2015 at 4:54 PM, Lei Zhang wrote: > Thank for your reply. > > I read that thread, but it just throw a exception when using confused > params. > > To solve this issue, is it worth to create a now micro version of api to > implement above cli? > As i see at the last (May, 13) Matt's comment in https://review.openstack.org/#/c/154068/ some work is planned or even started. I do not know details. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Removing BDM devices from POST requests
Device merging is used by EC2 API at least. This allows us (EC2 API team) to implement a use case of changing attached volume parameters when launching an instance from a volume backed image. If the image's bdms contain several volume descriptions, but a user wants to change some of them for a new instance, he can [1] E.g. the user may want to increase root volume size, skip another volume. Nova API doesn't support this ability fine (we must specify full bdm rather than changes only), but it works at least. If you remove device names from API, what alternative way are you going to provide instead for this purpose? [1] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html#Using_OverridingAMIBDM Thanks, Feodor Tersin. From: Matt Riedemann Sent: Sunday, April 2, 2017 6:59:12 PM To: Matt Riedemann; openstack-dev@lists.openstack.org; openstack-operat...@lists.openstack.org Subject: Re: [openstack-dev] [nova] Removing BDM devices from POST requests On 4/2/2017 10:13 AM, Matt Riedemann wrote: > On 4/1/2017 1:01 PM, Matt Riedemann wrote: >> I know we've talked about this over and over and another bug [1] >> reminded me of it. We have long talked about removing the ability to >> specify a block device name when creating a server or attaching a volume >> because we can't honor the requested device name anyway and trying to do >> so just causes issues. That's part of the reason why the libvirt driver >> stopped honoring the block device name in requests back in Liberty [2]. >> >> I think we all agree on removing the device name from the API, but I'm >> having a hard time remembering if someone signed up to write a spec for >> this. I could have sworn this came up recently and someone said they'd >> write a spec, but I can't remember. >> >> So this is my attempt at remembering and if it's all a dream, then is >> anyone interested in owning this? If not, I'll start writing the spec >> this week. >> >> [1] https://bugs.launchpad.net/nova/+bug/1648323 >> > > One thing I thought about with this is if you want to identify the block > device after it's attached to the server, you can use device tags to do > that (introduced in microversion 2.32). But that's only at server create > time until [1] is merged. So removing the device from attach volume > requests is probably dependent on completing Artom's blueprint first for > tagging volumes when you attach them to an existing server. > > [1] > https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/virt-device-tagged-attach-detach.html > > Spec started here: https://review.openstack.org/#/c/452546/ -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Removing BDM devices from POST requests
> Can you give some more details? How does this actually "merge" or > replace BDMs defined in the image metadata? Is this because we use > device name as part of a hack for a primary key in the database [1]? I > assume it is. The lack of unique constraint in the BDM data model has > also always been a source of bugs, especially with cells v1 and why we > talk about adding a uuid column to that table every few months. Right. This is not pure merging, but replacing, based on device names as a primary key. Important point is that it is a user defined, human readable key. > I've never actually heard of this use case but it looks like it's been > baked in since the legacy BDM behavior in the code, which is itself a > bunch of technical debt that I'd love to remove at some point. A couple of years ago when Dipanov accidentally broke the merging, we asked him to fix it here in ML, and he did it. If it's important, i'll find these reviews and ML thread. > I guess what I'm trying to get at is, is this just important for EC2 > compatibility? Or do non-AWS OpenStack users care about this use case, > because we definitely don't advertise this anywhere in our > documentation, or test it in any of our integrated testing (Tempest). So > just because it happens to work by chance of a poor data model doesn't > make me want to bend over backward to keep this working. Well, i cannot estimate the importance in absolute measurement, but in comparison with OpenStack this use case is more important in AWS. Volume backed images (EBS images) are used in AWS much more widely than in OpenStack. There are some difficulties in Nova and Cinder because that users try to avoid using volume backed images in favor of disk based (instance-store) ones. This explain why this use case is less important for pure OpenStack users. > If we wanted to support updating/overriding a specific image BDM during > server create, I'd think we could do something more straight-forward in > the compute API, like add an "image_bdm_override=True" field to > block_device_mapping_v2, something like that. What do you think about > that alternative? Since you want to delete the only natural key from bdm, how to refer on a certain bdm in parameters? Honestly, without real merging implemented in Nova, such workarounds fused into bdm structure do not look good for me. It looks like a crutch for something unfinished. Another disadvantages is that this new field may be added into DB, etc., because all other bdm fields are. Perhaps more appropriate way is to add a new 'ignore_image_bdms' parameter to run instance method. This brings another questions, but does not directly affect bdms at least. Thanks, Feodor Tersin. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Removing BDM devices from POST requests
> I raised the question though, could we move forward with removing the
> device field from the "POST /servers/{server_id}/os-volume_attachments"
> API since that doesn't do anything with image BDM overrides.
Will this affect detaching/attaching of root volume
(https://review.openstack.org/#/c/340874/2/specs/ocata/approved/detach-boot-volume.rst)?
I do not know if and how the spec is implemented, but it says:
---
The usual attach volume API call will be used to attach the boot volume.
The volume attach operation allows a user to specify the name of the device.
The boot device name of an instance is known so that is used to determine
that the user is attempting to attach the volume as the root device.
---
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Fix evaluation of host disk usage by volume-backed instances
Hi Abhishek There were a number of tries to analyze if an instance is booted from volume in DiskFilter itself. All reviews were stopped by core team members. As for you assume about image_ref is None for such instances, it is not right for an instance booted from a volume backed image (a snapshot of another volume backed instance). In this case image_ref refers to the snapshot - Glance image which does not contain data, but which metadata has links to volume snapshot(s). You also need to keep in mind that many other parts of code uses root_gb directly. E.g. nova-manage script calculates size of used host space by it (at least it did that a year ago). Another thing (why i stopped to work on it) is that this change (setting root_gb to 0) does not fix resize case. In this case scheduler and other components must not check root_gb, but must do checks with corresponded attribute of a new flavor. And i did not found an easy way how to provide there the fact that root_gb need to be ignored. I hope these info was useful for you. Thanks, Feodor Tersin From: Kekane, Abhishek Sent: Friday, August 12, 2016 4:29:11 PM To: OpenStack Development Mailing List (openstack-dev@lists.openstack.org) Subject: [openstack-dev] [nova] Fix evaluation of host disk usage by volume-backed instances Hi Nova developers, This is about the patch: https://review.openstack.org/#/c/200870/19 We would like to fix this issue in Newton and back port it to Mitaka. Reason: Ubuntu 16.04 LTS supports Mitaka release. If we wait for this fix until Ocata release (~April 2017), then Ubuntu team might need some more time to release Ocata in 16.04 (~Oct 2017). I think it will be too late to fix such an important and critical issue. Now on the other hand, if we fix this issue in Newton and back port it to Mitaka, the chances of getting this fix in Ubuntu 16.04 increases and it would be available to the Ubuntu users anytime between Oct and Dec of this year. We admit that this patch is a hack but considering its severity, it’s important to get it fixed as early as possible. Moreover, this code has been reviewed by many eyes so far and I don’t see its breaking current functionality. After this issue is fixed in the Ocata release during resource-providers implementation, we can delete these changes. This issue is discussed in Thu Aug 11 14:00:18 2016 UTC Nova meeting [1] and community came to conclusion that: We need to fix this issue in Newton but 1. Not willing to modify instance root_gb that is stored in instances db table. 2. Suggested to fix this issue in RT but that won't solve the scheduler DiskFilter issue completely. We have following approach in mind: 1. Scheduler DiskFilter should ignore root_gb from RequestSpec if instance is booted from volume. IMO boot server doesn't accept both image_id and volume_id to launch a new server. That means, if the instance is booted from volume, image_ref will always be None in the instances db table. i.e. instance.image_ref should be None. So, in the RequestSpec class, we should add an attribute "is_volume_backed' and set it to True when image is None. The Diskfilter has access to spec_obj, so simply check if is_volume_backed is True, if yes, ignore root_gb else count root_gb and take further action. This will solve the scheduler DiskFilter issue. 2. Resource tracker should also ignore root_gb while updating compute disk metrics. Again in "_get_usage_dict" method of resource_tracker.py, check if image is None, if yes, simply set root_gb to 0. This way each compute node will report disk metrics to the scheduler correctly. So the entire logic is based on image_ref of instance, it should be None if instance is booted from volume. I am working on a POC with this approach and will test all possible scenarios (boot, resize, reboot, compute service stop/start, shelved-unshelved etc). Please let me know your opinion about the same or you have any other solution in mind. [1] http://eavesdrop.openstack.org/meetings/nova/2016/nova.2016-08-11-14.00.log.html Thank you, Abhishek Kekane __ Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] How boot with bdm_v2={source_type=image, destination_type=local} should been used?
HI Zhenyu
You're referring to API doc, but you're checking it with nova cli, which does
not do exactly that what you want. In the case when you specify image and
block-device parameters, nova cli sends image_ref and two (!) bdms. Both these
bdms have the same boot_index and conflict with each other.
AFAIK nova cli adds the bdm for specified image_ref (not always, though).
In other words if you want to check API in this case, use pure curl instead of
nova cli to avoid these implicit behavior. Or you may use write tests on Python
using nova client.
From: Zhenyu Zheng
Sent: Wednesday, February 8, 2017 12:47:08 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Nova] How boot with bdm_v2={source_type=image,
destination_type=local} should been used?
Hi All,
As I was working on "Check destination_type when booting with bdm provided":
https://review.openstack.org/#/c/402372/ and addressing reviewers comments, I
find out that the current source_type=image, destination_type=local seems
unusable.
According to docs:
http://docs.openstack.org/developer/nova/block_device_mapping.html#block-device-mapping-v2,
it seems to me that "image --> local" means "boot from image", and as the doc
says, I should also provide image_ref param, but if I do so, Error raised:
ERROR (BadRequest): Block Device Mapping is Invalid: Boot sequence for the
instance and image/block device mapping combination is not valid. (HTTP 400)
(Request-ID: req-f848c6c1-0961-46c4-ac51-713fde042215)
If I just use bdm, it goes:
2017-02-08 11:04:24.929 24141 ERROR nova.compute.manager [instance:
6e44cafd-b330-4a10-8c77-eac60d58f20c] ImageNotFound: Image could not be found.
turned out we use '' as image ID to fetch image from glance, and obviously we
cannot get it.
Detailed Log and explaination could be found in my bug report:
https://bugs.launchpad.net/nova/+bug/1662748
So, what do we expect for this API usage?
Thanks,
Kevin Zheng
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] How boot with bdm_v2={source_type=image, destination_type=local} should been used?
Zhenyu,
btw iianm if you specify image only (w/o bdm) in the command line, nova cli
sends exactly that combination which you're interesting for: image_ref and
bdmv2.
From: Feodor Tersin
Sent: Wednesday, February 8, 2017 2:11:05 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Nova] How boot with bdm_v2={source_type=image,
destination_type=local} should been used?
HI Zhenyu
You're referring to API doc, but you're checking it with nova cli, which does
not do exactly that what you want. In the case when you specify image and
block-device parameters, nova cli sends image_ref and two (!) bdms. Both these
bdms have the same boot_index and conflict with each other.
AFAIK nova cli adds the bdm for specified image_ref (not always, though).
In other words if you want to check API in this case, use pure curl instead of
nova cli to avoid these implicit behavior. Or you may use write tests on Python
using nova client.
From: Zhenyu Zheng
Sent: Wednesday, February 8, 2017 12:47:08 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Nova] How boot with bdm_v2={source_type=image,
destination_type=local} should been used?
Hi All,
As I was working on "Check destination_type when booting with bdm provided":
https://review.openstack.org/#/c/402372/ and addressing reviewers comments, I
find out that the current source_type=image, destination_type=local seems
unusable.
According to docs:
http://docs.openstack.org/developer/nova/block_device_mapping.html#block-device-mapping-v2,
it seems to me that "image --> local" means "boot from image", and as the doc
says, I should also provide image_ref param, but if I do so, Error raised:
ERROR (BadRequest): Block Device Mapping is Invalid: Boot sequence for the
instance and image/block device mapping combination is not valid. (HTTP 400)
(Request-ID: req-f848c6c1-0961-46c4-ac51-713fde042215)
If I just use bdm, it goes:
2017-02-08 11:04:24.929 24141 ERROR nova.compute.manager [instance:
6e44cafd-b330-4a10-8c77-eac60d58f20c] ImageNotFound: Image could not be found.
turned out we use '' as image ID to fetch image from glance, and obviously we
cannot get it.
Detailed Log and explaination could be found in my bug report:
https://bugs.launchpad.net/nova/+bug/1662748
So, what do we expect for this API usage?
Thanks,
Kevin Zheng
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
