Re: [openstack-dev] [nova] Removing BDM devices from POST requests
On 4/4/2017 10:07 PM, Feodor Tersin wrote:
I raised the question though, could we move forward with removing the
device field from the "POST /servers/{server_id}/os-volume_attachments"
API since that doesn't do anything with image BDM overrides.
Will this affect detaching/attaching of root volume
(https://review.openstack.org/#/c/340874/2/specs/ocata/approved/detach-boot-volume.rst)?
I do not know if and how the spec is implemented, but it says:
---
The usual attach volume API call will be used to attach the boot volume.
The volume attach operation allows a user to specify the name of the device.
The boot device name of an instance is known so that is used to determine
that the user is attempting to attach the volume as the root device.
---
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
That blueprint never got implemented nor is being re-proposed for Pike.
Having said that, I wasn't aware of that part of it. I suppose the idea
is to provide the device name of the root device and then the API would
validate that the matching BDM for the instance based on device name has
boot_index=0 and detach it, otherwise fail because it's not the root
device BDM.
Anyway, I've abandoned my spec for Pike. It's a distraction at this
point and we have other things that need to get done first for BDMs in
the API before we can consider removing device name in the requests.
--
Thanks,
Matt
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Removing BDM devices from POST requests
> I raised the question though, could we move forward with removing the
> device field from the "POST /servers/{server_id}/os-volume_attachments"
> API since that doesn't do anything with image BDM overrides.
Will this affect detaching/attaching of root volume
(https://review.openstack.org/#/c/340874/2/specs/ocata/approved/detach-boot-volume.rst)?
I do not know if and how the spec is implemented, but it says:
---
The usual attach volume API call will be used to attach the boot volume.
The volume attach operation allows a user to specify the name of the device.
The boot device name of an instance is known so that is used to determine
that the user is attempting to attach the volume as the root device.
---
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Removing BDM devices from POST requests
On 4/2/2017 10:10 PM, Matt Riedemann wrote:
On 4/2/2017 8:08 PM, Feodor Tersin wrote:
Can you give some more details? How does this actually "merge" or
replace BDMs defined in the image metadata? Is this because we use
device name as part of a hack for a primary key in the database [1]? I
assume it is. The lack of unique constraint in the BDM data model has
also always been a source of bugs, especially with cells v1 and why we
talk about adding a uuid column to that table every few months.
Right. This is not pure merging, but replacing, based on device names as
a primary key. Important point is that it is a user defined, human
readable key.
I've never actually heard of this use case but it looks like it's been
baked in since the legacy BDM behavior in the code, which is itself a
bunch of technical debt that I'd love to remove at some point.
A couple of years ago when Dipanov accidentally broke the merging, we
asked him to fix it here in ML, and he did it. If it's important, i'll
find these reviews and ML thread.
The history isn't as important to me. What's frustrating is this is
something people care about, and has been regressed before, and we don't
have good enough test coverage or documentation to know it's something
people do care about. I think before we move forward with any of this,
we'll want a test added to Tempest that validates this use case so we
avoid regressing it again.
I guess what I'm trying to get at is, is this just important for EC2
compatibility? Or do non-AWS OpenStack users care about this use case,
because we definitely don't advertise this anywhere in our
documentation, or test it in any of our integrated testing (Tempest). So
just because it happens to work by chance of a poor data model doesn't
make me want to bend over backward to keep this working.
Well, i cannot estimate the importance in absolute measurement, but in
comparison with OpenStack this use case is more important in AWS. Volume
backed images (EBS images) are used in AWS much more widely than in
OpenStack. There are some difficulties in Nova and Cinder because that
users try to avoid using volume backed images in favor of disk based
(instance-store) ones. This explain why this use case is less important
for pure OpenStack users.
If we wanted to support updating/overriding a specific image BDM during
server create, I'd think we could do something more straight-forward in
the compute API, like add an "image_bdm_override=True" field to
block_device_mapping_v2, something like that. What do you think about
that alternative?
Since you want to delete the only natural key from bdm, how to refer on
a certain bdm in parameters? Honestly, without real merging implemented
in Nova, such workarounds fused into bdm structure do not look good for
me. It looks like a crutch for something unfinished. Another
disadvantages is that this new field may be added into DB, etc., because
all other bdm fields are. Perhaps more appropriate way is to add a new
'ignore_image_bdms' parameter to run instance method. This brings
another questions, but does not directly affect bdms at least.
I'm not sure what the best design is for maintaining this in the API
without using the device name. I rattled off a few options in the spec
review, but they aren't great options, and some simply won't work for
older BDM or image resources, so are non-starters. I'm hoping some
others with a fresh perspective on this can chime in with ideas,
otherwise I'll bring it up in the nova API subteam meeting this week so
we can talk about the best way forward.
Thanks again Feodor.
mdbooth raised a good point in the spec about the device just being a
convenient tag essentially for correlating the values for the image bdm
override during server create. I was thinking, since we have tags for
BDMs now as of the 2.32 microversion, couldn't we rely on those for the
image bdm override at some point?
Granted, we need to do two other things first:
1. Allow tagging BDMs during volume attach (Artom has a series for that
in Pike already).
2. Expose BDM tags out of the REST API (I have a spec for this but it's
getting caught up in also exposing VIF tags, which depends on what we
decide to do with os-virtual-interfaces and the os-interface APIs).
--
I raised the question though, could we move forward with removing the
device field from the "POST /servers/{server_id}/os-volume_attachments"
API since that doesn't do anything with image BDM overrides.
--
Thanks,
Matt
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Removing BDM devices from POST requests
On 4/2/2017 8:08 PM, Feodor Tersin wrote: Can you give some more details? How does this actually "merge" or replace BDMs defined in the image metadata? Is this because we use device name as part of a hack for a primary key in the database [1]? I assume it is. The lack of unique constraint in the BDM data model has also always been a source of bugs, especially with cells v1 and why we talk about adding a uuid column to that table every few months. Right. This is not pure merging, but replacing, based on device names as a primary key. Important point is that it is a user defined, human readable key. I've never actually heard of this use case but it looks like it's been baked in since the legacy BDM behavior in the code, which is itself a bunch of technical debt that I'd love to remove at some point. A couple of years ago when Dipanov accidentally broke the merging, we asked him to fix it here in ML, and he did it. If it's important, i'll find these reviews and ML thread. The history isn't as important to me. What's frustrating is this is something people care about, and has been regressed before, and we don't have good enough test coverage or documentation to know it's something people do care about. I think before we move forward with any of this, we'll want a test added to Tempest that validates this use case so we avoid regressing it again. I guess what I'm trying to get at is, is this just important for EC2 compatibility? Or do non-AWS OpenStack users care about this use case, because we definitely don't advertise this anywhere in our documentation, or test it in any of our integrated testing (Tempest). So just because it happens to work by chance of a poor data model doesn't make me want to bend over backward to keep this working. Well, i cannot estimate the importance in absolute measurement, but in comparison with OpenStack this use case is more important in AWS. Volume backed images (EBS images) are used in AWS much more widely than in OpenStack. There are some difficulties in Nova and Cinder because that users try to avoid using volume backed images in favor of disk based (instance-store) ones. This explain why this use case is less important for pure OpenStack users. If we wanted to support updating/overriding a specific image BDM during server create, I'd think we could do something more straight-forward in the compute API, like add an "image_bdm_override=True" field to block_device_mapping_v2, something like that. What do you think about that alternative? Since you want to delete the only natural key from bdm, how to refer on a certain bdm in parameters? Honestly, without real merging implemented in Nova, such workarounds fused into bdm structure do not look good for me. It looks like a crutch for something unfinished. Another disadvantages is that this new field may be added into DB, etc., because all other bdm fields are. Perhaps more appropriate way is to add a new 'ignore_image_bdms' parameter to run instance method. This brings another questions, but does not directly affect bdms at least. I'm not sure what the best design is for maintaining this in the API without using the device name. I rattled off a few options in the spec review, but they aren't great options, and some simply won't work for older BDM or image resources, so are non-starters. I'm hoping some others with a fresh perspective on this can chime in with ideas, otherwise I'll bring it up in the nova API subteam meeting this week so we can talk about the best way forward. Thanks again Feodor. -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Removing BDM devices from POST requests
> Can you give some more details? How does this actually "merge" or > replace BDMs defined in the image metadata? Is this because we use > device name as part of a hack for a primary key in the database [1]? I > assume it is. The lack of unique constraint in the BDM data model has > also always been a source of bugs, especially with cells v1 and why we > talk about adding a uuid column to that table every few months. Right. This is not pure merging, but replacing, based on device names as a primary key. Important point is that it is a user defined, human readable key. > I've never actually heard of this use case but it looks like it's been > baked in since the legacy BDM behavior in the code, which is itself a > bunch of technical debt that I'd love to remove at some point. A couple of years ago when Dipanov accidentally broke the merging, we asked him to fix it here in ML, and he did it. If it's important, i'll find these reviews and ML thread. > I guess what I'm trying to get at is, is this just important for EC2 > compatibility? Or do non-AWS OpenStack users care about this use case, > because we definitely don't advertise this anywhere in our > documentation, or test it in any of our integrated testing (Tempest). So > just because it happens to work by chance of a poor data model doesn't > make me want to bend over backward to keep this working. Well, i cannot estimate the importance in absolute measurement, but in comparison with OpenStack this use case is more important in AWS. Volume backed images (EBS images) are used in AWS much more widely than in OpenStack. There are some difficulties in Nova and Cinder because that users try to avoid using volume backed images in favor of disk based (instance-store) ones. This explain why this use case is less important for pure OpenStack users. > If we wanted to support updating/overriding a specific image BDM during > server create, I'd think we could do something more straight-forward in > the compute API, like add an "image_bdm_override=True" field to > block_device_mapping_v2, something like that. What do you think about > that alternative? Since you want to delete the only natural key from bdm, how to refer on a certain bdm in parameters? Honestly, without real merging implemented in Nova, such workarounds fused into bdm structure do not look good for me. It looks like a crutch for something unfinished. Another disadvantages is that this new field may be added into DB, etc., because all other bdm fields are. Perhaps more appropriate way is to add a new 'ignore_image_bdms' parameter to run instance method. This brings another questions, but does not directly affect bdms at least. Thanks, Feodor Tersin. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Removing BDM devices from POST requests
On 4/2/2017 12:35 PM, Feodor Tersin wrote: Device merging is used by EC2 API at least. This allows us (EC2 API team) to implement a use case of changing attached volume parameters when launching an instance from a volume backed image. If the image's bdms contain several volume descriptions, but a user wants to change some of them for a new instance, he can [1] E.g. the user may want to increase root volume size, skip another volume. Nova API doesn't support this ability fine (we must specify full bdm rather than changes only), but it works at least. If you remove device names from API, what alternative way are you going to provide instead for this purpose? [1] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html#Using_OverridingAMIBDM Thanks, Feodor Tersin. Can you give some more details? How does this actually "merge" or replace BDMs defined in the image metadata? Is this because we use device name as part of a hack for a primary key in the database [1]? I assume it is. The lack of unique constraint in the BDM data model has also always been a source of bugs, especially with cells v1 and why we talk about adding a uuid column to that table every few months. I've never actually heard of this use case but it looks like it's been baked in since the legacy BDM behavior in the code, which is itself a bunch of technical debt that I'd love to remove at some point. I guess what I'm trying to get at is, is this just important for EC2 compatibility? Or do non-AWS OpenStack users care about this use case, because we definitely don't advertise this anywhere in our documentation, or test it in any of our integrated testing (Tempest). So just because it happens to work by chance of a poor data model doesn't make me want to bend over backward to keep this working. If we wanted to support updating/overriding a specific image BDM during server create, I'd think we could do something more straight-forward in the compute API, like add an "image_bdm_override=True" field to block_device_mapping_v2, something like that. What do you think about that alternative? [1] https://github.com/openstack/nova/blob/468916ee57fede4d0c89fbf9c776269943e2cb44/nova/db/sqlalchemy/api.py#L4170-L4175 -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Removing BDM devices from POST requests
Device merging is used by EC2 API at least. This allows us (EC2 API team) to implement a use case of changing attached volume parameters when launching an instance from a volume backed image. If the image's bdms contain several volume descriptions, but a user wants to change some of them for a new instance, he can [1] E.g. the user may want to increase root volume size, skip another volume. Nova API doesn't support this ability fine (we must specify full bdm rather than changes only), but it works at least. If you remove device names from API, what alternative way are you going to provide instead for this purpose? [1] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html#Using_OverridingAMIBDM Thanks, Feodor Tersin. From: Matt Riedemann Sent: Sunday, April 2, 2017 6:59:12 PM To: Matt Riedemann; openstack-dev@lists.openstack.org; openstack-operat...@lists.openstack.org Subject: Re: [openstack-dev] [nova] Removing BDM devices from POST requests On 4/2/2017 10:13 AM, Matt Riedemann wrote: > On 4/1/2017 1:01 PM, Matt Riedemann wrote: >> I know we've talked about this over and over and another bug [1] >> reminded me of it. We have long talked about removing the ability to >> specify a block device name when creating a server or attaching a volume >> because we can't honor the requested device name anyway and trying to do >> so just causes issues. That's part of the reason why the libvirt driver >> stopped honoring the block device name in requests back in Liberty [2]. >> >> I think we all agree on removing the device name from the API, but I'm >> having a hard time remembering if someone signed up to write a spec for >> this. I could have sworn this came up recently and someone said they'd >> write a spec, but I can't remember. >> >> So this is my attempt at remembering and if it's all a dream, then is >> anyone interested in owning this? If not, I'll start writing the spec >> this week. >> >> [1] https://bugs.launchpad.net/nova/+bug/1648323 >> > > One thing I thought about with this is if you want to identify the block > device after it's attached to the server, you can use device tags to do > that (introduced in microversion 2.32). But that's only at server create > time until [1] is merged. So removing the device from attach volume > requests is probably dependent on completing Artom's blueprint first for > tagging volumes when you attach them to an existing server. > > [1] > https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/virt-device-tagged-attach-detach.html > > Spec started here: https://review.openstack.org/#/c/452546/ -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Removing BDM devices from POST requests
On 4/2/2017 10:13 AM, Matt Riedemann wrote: On 4/1/2017 1:01 PM, Matt Riedemann wrote: I know we've talked about this over and over and another bug [1] reminded me of it. We have long talked about removing the ability to specify a block device name when creating a server or attaching a volume because we can't honor the requested device name anyway and trying to do so just causes issues. That's part of the reason why the libvirt driver stopped honoring the block device name in requests back in Liberty [2]. I think we all agree on removing the device name from the API, but I'm having a hard time remembering if someone signed up to write a spec for this. I could have sworn this came up recently and someone said they'd write a spec, but I can't remember. So this is my attempt at remembering and if it's all a dream, then is anyone interested in owning this? If not, I'll start writing the spec this week. [1] https://bugs.launchpad.net/nova/+bug/1648323 One thing I thought about with this is if you want to identify the block device after it's attached to the server, you can use device tags to do that (introduced in microversion 2.32). But that's only at server create time until [1] is merged. So removing the device from attach volume requests is probably dependent on completing Artom's blueprint first for tagging volumes when you attach them to an existing server. [1] https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/virt-device-tagged-attach-detach.html Spec started here: https://review.openstack.org/#/c/452546/ -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Removing BDM devices from POST requests
On 4/1/2017 1:01 PM, Matt Riedemann wrote: I know we've talked about this over and over and another bug [1] reminded me of it. We have long talked about removing the ability to specify a block device name when creating a server or attaching a volume because we can't honor the requested device name anyway and trying to do so just causes issues. That's part of the reason why the libvirt driver stopped honoring the block device name in requests back in Liberty [2]. I think we all agree on removing the device name from the API, but I'm having a hard time remembering if someone signed up to write a spec for this. I could have sworn this came up recently and someone said they'd write a spec, but I can't remember. So this is my attempt at remembering and if it's all a dream, then is anyone interested in owning this? If not, I'll start writing the spec this week. [1] https://bugs.launchpad.net/nova/+bug/1648323 Oops I forgot the reference for the Liberty change. [2] https://review.openstack.org/#/c/189632/ -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Removing BDM devices from POST requests
On 4/1/2017 1:01 PM, Matt Riedemann wrote: I know we've talked about this over and over and another bug [1] reminded me of it. We have long talked about removing the ability to specify a block device name when creating a server or attaching a volume because we can't honor the requested device name anyway and trying to do so just causes issues. That's part of the reason why the libvirt driver stopped honoring the block device name in requests back in Liberty [2]. I think we all agree on removing the device name from the API, but I'm having a hard time remembering if someone signed up to write a spec for this. I could have sworn this came up recently and someone said they'd write a spec, but I can't remember. So this is my attempt at remembering and if it's all a dream, then is anyone interested in owning this? If not, I'll start writing the spec this week. [1] https://bugs.launchpad.net/nova/+bug/1648323 One thing I thought about with this is if you want to identify the block device after it's attached to the server, you can use device tags to do that (introduced in microversion 2.32). But that's only at server create time until [1] is merged. So removing the device from attach volume requests is probably dependent on completing Artom's blueprint first for tagging volumes when you attach them to an existing server. [1] https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/virt-device-tagged-attach-detach.html -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Removing BDM devices from POST requests
I know we've talked about this over and over and another bug [1] reminded me of it. We have long talked about removing the ability to specify a block device name when creating a server or attaching a volume because we can't honor the requested device name anyway and trying to do so just causes issues. That's part of the reason why the libvirt driver stopped honoring the block device name in requests back in Liberty [2]. I think we all agree on removing the device name from the API, but I'm having a hard time remembering if someone signed up to write a spec for this. I could have sworn this came up recently and someone said they'd write a spec, but I can't remember. So this is my attempt at remembering and if it's all a dream, then is anyone interested in owning this? If not, I'll start writing the spec this week. [1] https://bugs.launchpad.net/nova/+bug/1648323 -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
