Nikola, thank you for the explanation.
So here is what we have problems with now (I might not be completely
correct in some facts and guesses though):
The following review made it worse for us initially:
https://review.openstack.org/#/c/59034/25
As I understand in this review you replaced storing of bdmv1 with bdmv2.
And the latter stopped storing the device name into snapshot.
First of all, the question is, how do you guarantee that the order of
devices (not boot order but the vda, vdb, vdc... naming rather) will
stay the same after restoring instance from such a snapshot. You must
rely on an order they are stored in the image with respect to boot
devices, right?
Another question, did it work for Xen fully before (honoring the names,
setting them properly inside instance and reporting them)? And if it
did, now with bdmv2 it did stop, right? If that's the case, is it the
desired behavior?
For EC2 API, I presume, ideal situation would be if all backends could
support device names fully. Since it is unrealistic, I guess, we'd still
aim for it working fully where possible (Xen), setting up, assigning the
same names to devices inside instance, i.e. complete synchronization of
the outside and inside world. And for the rest to work in the most
predictable way.
Do you think, since you're replicating the libvirt logic to order
devices, you could give them some ordered names (a, b, c, d) according
to the order and report them through public APIs and store into
snapshots? This way we could use them to address these same devices in
our use cases. And in case of Xen you can use real names instead.
For the reference, we have the following use cases concerned with device
names (I'll be using EC2 API commands for use cases - they should be
pretty much self-explanative, but you can look them up by the links for
more descriptions):
1. RunInstance. Change parameters of devices during instance booting
from image. In Grizzly it worked so we could specify changed BDM in
parameters, it overwrote in nova DB the one coming from image and then
started the instance with new parameters. The only key for addressing
devices in this use case is the very device name. And now we don't have
it for the volumes in BDM coming from the image, because nova stopped
putting this information into the image.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html
2. Devices names for Xen-backed instances to work fully. We should be
able to specify required device names during initial instance creation,
they should be stored into an image when the instance is shapshotted, we
can fetch info and change parameters of such volume during subsequent
operations, and the device names inside the instance should be named
exactly.
3. DescribeInstances and DescribeInstanceAttributes to return BDM with
device names ideally corresponding to actual device naming in instance.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstanceAttribute.html
4. DescribeImages and DescribeImageAttributes to return BDM with device
names ideally corresponding to the ones in instance before snapshotting.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeImages.html
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeImageAttribute.html
5. AttachVolume with the specified device name.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_AttachVolume.html
6. ModifyInstanceAttribute with BDM as parameter.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_ModifyInstanceAttribute.html
7. ModifyImageAttribute with BDM as parameter.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_ModifyImageAttribute.html
Best regards,
Alex Levine
On 5/27/15 8:55 PM, Nikola Đipanov wrote:
On 05/27/2015 09:47 AM, Alexandre Levine wrote:
Hi all,
I'd like to bring up this matter again, although it was at some extent
discussed during the recent summit.
The problem arises from the fact that the functionality exposing device
names for usage through public APIs is deteriorating in nova. It's being
deliberately removed because as I understand, it doesn't universally and
consistently work in all of the backends. It happens since IceHouse and
introduction of bdm v2. The following very recent review is one of the
ongoing efforts in this direction:
https://review.openstack.org/#/c/185438/
I've abandoned the change as it is clear we need to discuss how to go
about this some more.
But first let me try to give a bit more detailed explanation and
background to what the deal is with device names. Supplying device names
that will be honoured by the guests is really only possible with Xen PV
guests (meaning the guest needs to be running PV-enabled kernel and
drivers).
Back in Havana, when we were working on [1] (see [2] for more details)
the basic idea was that we will still accept device names because
removing them from the public API is not likely to happen (mostly
because of the EC2 compatibility), but in case of libvirt driver, we
will treat them as hints only, and provide our own (by mostly
replicating the logic libvirt uses to order devices [3]). We also
allowed for device names to not be specified by the user as this is
really what anyone not using the EC2 API should be doing (users using
the EC2 API do however need to be aware the fact that i may not be
honoured).
[1]
https://blueprints.launchpad.net/nova/+spec/improve-block-device-handling
[2] https://wiki.openstack.org/wiki/BlockDeviceConfig
[3]
https://github.com/openstack/nova/blob/master/nova/virt/libvirt/blockinfo.py
The reason for my concern is that EC2 API have some important cases
relying on this information (some of them have no workarounds). Namely:
1. Change of parameters set by image for instance booting.
2. Showing instance's devices information by euca2ools.
3. Providing additional volumes for instance booting
4. Attaching volume
etc...
So based on the above - it seems to me that you think we are removing
the information about device names completely. That's not the case -
currently it is simply not mandatory for the Nova boot API call (it was
never mandatory for volume attach afaict) - you can still pass it in,
though libvirt may not honour it. It will still be tracked by the Nova
DB and available for users to refer to.
Related to device names and additional related features we have troubles
with now:
1. All device name related features
As I said - they are not removed, in addition, you can still completely
disregard the BDMv2 syntax as Nova should transparently handle old-style
syntax when passed in (actually since BDM info is stored with images
when snapshotting and it may have been v1 syntax - it is likely that we
will never remove this support). If you are seeing some bugs related to
this - please report them.
2. Modification of deleteOnTermination flag
I don't have enough details on this but if some behaviour has changed
when using the old syntax - it is likely a bug so please report it.
3. Modification of parameters for instance booting
Again - I am not sure what this is related to exactly - but none of the
parameters have changed really (only new ones were added). It would be
good to get more information on this (preferably a bug report).
4. deleteOnTermination and size of volume aren't stored into instance
snapshots now.
This does sound like a bug - and hopefully an easy to fix one.
Discussions during the summit on the matter were complicated because
nobody present really understood in details why and what is happening
with this functionality in nova. It was decided though, that overall
direction would be to add necessary features or restore them unless
there is something really showstopping:
https://etherpad.openstack.org/p/YVR-nova-contributor-meetup
As I understand, Nikola Depanov is the one working on the matter for
some time obviously is the best person who can help to resolve the
situation. Nikola, if possible, could you help with it and clarify the
issue.
My suggestion, based on my limited knowledge at the moment, still is to
restore back or add all of the necessary APIs and provide tickets or
known issues for the cases where the functionality is suffering from the
backend limitations.
Please let me know what you think.
As explained above - nothing was intentionally removed, and if something
broke - it's a bug that we should fix, so I urge the team behind the EC2
API on stackforge to report those, and I will try to at least look into
them, if not fix them. We might want to have a tag for EC2 related bugs
in LP (I seem to remember there being such a thing before).
Device names though are not something we can easily resolve without
having the users of the EC2 API be aware that they may be talking to a
Nova deployment, however I am unsure how much of a problem this is in
practice, as I believe most users would have to be anyway.
Hope this lifts some of the mystery around the topic - and I look
forward to helping the EC2 efforts - but we really need to make sure we
have the issues written down first :)
N.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
