Re: [openstack-dev] [nova] Create an instance with a custom uuid
Hi all, while I was working on blueprint Pasquale addressed this issue. In the use case explained both features are involved but the user defined UUID does not depend strictly on original blueprint but it can be a new one. If there is no objection I could write a blueprint/specification regarding to last solution. Any thought? BR, Angelo On 01/10/2014 18:07, Joe Gordon wrote: On Wed, Oct 1, 2014 at 8:29 AM, Solly Ross sr...@redhat.com mailto:sr...@redhat.com wrote: (response inline) - Original Message - From: Pasquale Porreca pasquale.porr...@dektech.com.au mailto:pasquale.porr...@dektech.com.au To: openstack-dev@lists.openstack.org mailto:openstack-dev@lists.openstack.org Sent: Wednesday, October 1, 2014 11:08:50 AM Subject: Re: [openstack-dev] [nova] Create an instance with a custom uuid Thank you for the answers. I understood the concerns about having the UUID completely user defined and I also understand Nova has no interest in supporting a customized algorithm to generate UUID. Anyway I may have found a solution that will cover my use case and respect the standard for UUID (RFC 4122 http://www.ietf.org/rfc/rfc4122.txt ) . The generation of the UUID in Nova make use of the function uuid4() from the module uuid.py to have an UUID (pseudo)random, according to version 4 described in RFC 4122. Anyway this is not the only algorithm supported in the standard (and implemented yet in uuid.py ). In particular I focused my attention on UUID version 1 and the method uuid1(node=None, clock_seq=None) that allows to pass as parameter a part of the UUID ( node is the field containing the last 12 hexadecimal digits of the UUID). So my idea was to give the chance to the user to set uiid version (1 or 4, with the latter as default) when creating a new instance and in case of version 1 to pass optionally a value for parameter node . I would think that we could just have a node parameter here, and automatically use version 1 if that parameter is passed (if we decided to go the route of changing the current UUID behavior). From what I gather this requested change in API is based on for your blueprint https://blueprints.launchpad.net/nova/+spec/pxe-boot-instance. Since your blueprint is not approved yet discussing further work to improve it is a bit premature. Any thoughts? On 09/30/14 14:07, Andrew Laski wrote: On 09/30/2014 06:53 AM, Pasquale Porreca wrote: Going back to my original question, I would like to know: 1) Is it acceptable to have the UUID passed from client side? In my opinion, no. This opens a door to issues we currently don't need to deal with, and use cases I don't think Nova should support. Another possibility, which I don't like either, would be to pass in some data which could influence the generation of the UUID to satisfy requirements. But there was a suggestion to look into addressing your use case on the QEMU mailing list, which I think would be a better approach. 2) What is the correct way to do it? I started to implement this feature, simply passing it as metadata with key uuid, but I feel that this feature should have a reserved option rather then use metadata. On 09/25/14 17:26, Daniel P. Berrange wrote: On Thu, Sep 25, 2014 at 05:23:22PM +0200, Pasquale Porreca wrote: This is correct Daniel, except that that it is done by the virtual firmware/BIOS of the virtual machine and not by the OS (not yet installed at that time). This is the reason we thought about UUID: it is yet used by the iPXE client to be included in Bootstrap Protocol messages, it is taken from the uuid field in libvirt template and the uuid in libvirt is set by OpenStack; the only missing passage is the chance to set the UUID in OpenStack instead to have it randomly generated. Having another user defined tag in libvirt won't help for our issue, since it won't be included in Bootstrap Protocol messages, not without changes in the virtual BIOS/firmware (as you stated too) and honestly my team doesn't have interest in this (neither the competence). I don't think the configdrive or metadata service would help either: the OS on the instance is not yet installed at that time (the target if the network boot is exactly to install the OS on the instance!), so it won't be able to mount it. Ok, yes, if we're considering the DHCP client inside the iPXE BIOS blob, then I don't see any currently viable options besides UUID. There's no mechanism
Re: [openstack-dev] [nova] Create an instance with a custom uuid
Thank you for the answers. I understood the concerns about having the UUID completely user defined and I also understand Nova has no interest in supporting a customized algorithm to generate UUID. Anyway I may have found a solution that will cover my use case and respect the standard for UUID (RFC 4122 http://www.ietf.org/rfc/rfc4122.txt) . The generation of the UUID in Nova make use of the function /uuid4()/ from the module /uuid.py/ to have an UUID (pseudo)random, according to version 4 described in RFC 4122. Anyway this is not the only algorithm supported in the standard (and implemented yet in /uuid.py/). In particular I focused my attention on UUID version 1 and the method /uuid1(node=None, clock_seq=None)/ that allows to pass as parameter a part of the UUID (/node/ is the field containing the last 12 hexadecimal digits of the UUID). So my idea was to give the chance to the user to set uiid version (1 or 4, with the latter as default) when creating a new instance and in case of version 1 to pass optionally a value for parameter /node/. Any thoughts? On 09/30/14 14:07, Andrew Laski wrote: On 09/30/2014 06:53 AM, Pasquale Porreca wrote: Going back to my original question, I would like to know: 1) Is it acceptable to have the UUID passed from client side? In my opinion, no. This opens a door to issues we currently don't need to deal with, and use cases I don't think Nova should support. Another possibility, which I don't like either, would be to pass in some data which could influence the generation of the UUID to satisfy requirements. But there was a suggestion to look into addressing your use case on the QEMU mailing list, which I think would be a better approach. 2) What is the correct way to do it? I started to implement this feature, simply passing it as metadata with key uuid, but I feel that this feature should have a reserved option rather then use metadata. On 09/25/14 17:26, Daniel P. Berrange wrote: On Thu, Sep 25, 2014 at 05:23:22PM +0200, Pasquale Porreca wrote: This is correct Daniel, except that that it is done by the virtual firmware/BIOS of the virtual machine and not by the OS (not yet installed at that time). This is the reason we thought about UUID: it is yet used by the iPXE client to be included in Bootstrap Protocol messages, it is taken from the uuid field in libvirt template and the uuid in libvirt is set by OpenStack; the only missing passage is the chance to set the UUID in OpenStack instead to have it randomly generated. Having another user defined tag in libvirt won't help for our issue, since it won't be included in Bootstrap Protocol messages, not without changes in the virtual BIOS/firmware (as you stated too) and honestly my team doesn't have interest in this (neither the competence). I don't think the configdrive or metadata service would help either: the OS on the instance is not yet installed at that time (the target if the network boot is exactly to install the OS on the instance!), so it won't be able to mount it. Ok, yes, if we're considering the DHCP client inside the iPXE BIOS blob, then I don't see any currently viable options besides UUID. There's no mechanism for passing any other data into iPXE that I am aware of, though if there is a desire todo that it could be raised on the QEMU mailing list for discussion. Regards, Daniel ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
On Wed, Oct 1, 2014 at 8:29 AM, Solly Ross sr...@redhat.com wrote: (response inline) - Original Message - From: Pasquale Porreca pasquale.porr...@dektech.com.au To: openstack-dev@lists.openstack.org Sent: Wednesday, October 1, 2014 11:08:50 AM Subject: Re: [openstack-dev] [nova] Create an instance with a custom uuid Thank you for the answers. I understood the concerns about having the UUID completely user defined and I also understand Nova has no interest in supporting a customized algorithm to generate UUID. Anyway I may have found a solution that will cover my use case and respect the standard for UUID (RFC 4122 http://www.ietf.org/rfc/rfc4122.txt ) . The generation of the UUID in Nova make use of the function uuid4() from the module uuid.py to have an UUID (pseudo)random, according to version 4 described in RFC 4122. Anyway this is not the only algorithm supported in the standard (and implemented yet in uuid.py ). In particular I focused my attention on UUID version 1 and the method uuid1(node=None, clock_seq=None) that allows to pass as parameter a part of the UUID ( node is the field containing the last 12 hexadecimal digits of the UUID). So my idea was to give the chance to the user to set uiid version (1 or 4, with the latter as default) when creating a new instance and in case of version 1 to pass optionally a value for parameter node . I would think that we could just have a node parameter here, and automatically use version 1 if that parameter is passed (if we decided to go the route of changing the current UUID behavior). From what I gather this requested change in API is based on for your blueprint https://blueprints.launchpad.net/nova/+spec/pxe-boot-instance. Since your blueprint is not approved yet discussing further work to improve it is a bit premature. Any thoughts? On 09/30/14 14:07, Andrew Laski wrote: On 09/30/2014 06:53 AM, Pasquale Porreca wrote: Going back to my original question, I would like to know: 1) Is it acceptable to have the UUID passed from client side? In my opinion, no. This opens a door to issues we currently don't need to deal with, and use cases I don't think Nova should support. Another possibility, which I don't like either, would be to pass in some data which could influence the generation of the UUID to satisfy requirements. But there was a suggestion to look into addressing your use case on the QEMU mailing list, which I think would be a better approach. 2) What is the correct way to do it? I started to implement this feature, simply passing it as metadata with key uuid, but I feel that this feature should have a reserved option rather then use metadata. On 09/25/14 17:26, Daniel P. Berrange wrote: On Thu, Sep 25, 2014 at 05:23:22PM +0200, Pasquale Porreca wrote: This is correct Daniel, except that that it is done by the virtual firmware/BIOS of the virtual machine and not by the OS (not yet installed at that time). This is the reason we thought about UUID: it is yet used by the iPXE client to be included in Bootstrap Protocol messages, it is taken from the uuid field in libvirt template and the uuid in libvirt is set by OpenStack; the only missing passage is the chance to set the UUID in OpenStack instead to have it randomly generated. Having another user defined tag in libvirt won't help for our issue, since it won't be included in Bootstrap Protocol messages, not without changes in the virtual BIOS/firmware (as you stated too) and honestly my team doesn't have interest in this (neither the competence). I don't think the configdrive or metadata service would help either: the OS on the instance is not yet installed at that time (the target if the network boot is exactly to install the OS on the instance!), so it won't be able to mount it. Ok, yes, if we're considering the DHCP client inside the iPXE BIOS blob, then I don't see any currently viable options besides UUID. There's no mechanism for passing any other data into iPXE that I am aware of, though if there is a desire todo that it could be raised on the QEMU mailing list for discussion. Regards, Daniel ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin
Re: [openstack-dev] [nova] Create an instance with a custom uuid
Going back to my original question, I would like to know: 1) Is it acceptable to have the UUID passed from client side? 2) What is the correct way to do it? I started to implement this feature, simply passing it as metadata with key uuid, but I feel that this feature should have a reserved option rather then use metadata. On 09/25/14 17:26, Daniel P. Berrange wrote: On Thu, Sep 25, 2014 at 05:23:22PM +0200, Pasquale Porreca wrote: This is correct Daniel, except that that it is done by the virtual firmware/BIOS of the virtual machine and not by the OS (not yet installed at that time). This is the reason we thought about UUID: it is yet used by the iPXE client to be included in Bootstrap Protocol messages, it is taken from the uuid field in libvirt template and the uuid in libvirt is set by OpenStack; the only missing passage is the chance to set the UUID in OpenStack instead to have it randomly generated. Having another user defined tag in libvirt won't help for our issue, since it won't be included in Bootstrap Protocol messages, not without changes in the virtual BIOS/firmware (as you stated too) and honestly my team doesn't have interest in this (neither the competence). I don't think the configdrive or metadata service would help either: the OS on the instance is not yet installed at that time (the target if the network boot is exactly to install the OS on the instance!), so it won't be able to mount it. Ok, yes, if we're considering the DHCP client inside the iPXE BIOS blob, then I don't see any currently viable options besides UUID. There's no mechanism for passing any other data into iPXE that I am aware of, though if there is a desire todo that it could be raised on the QEMU mailing list for discussion. Regards, Daniel -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
On 09/30/2014 06:53 AM, Pasquale Porreca wrote: Going back to my original question, I would like to know: 1) Is it acceptable to have the UUID passed from client side? In my opinion, no. This opens a door to issues we currently don't need to deal with, and use cases I don't think Nova should support. Another possibility, which I don't like either, would be to pass in some data which could influence the generation of the UUID to satisfy requirements. But there was a suggestion to look into addressing your use case on the QEMU mailing list, which I think would be a better approach. 2) What is the correct way to do it? I started to implement this feature, simply passing it as metadata with key uuid, but I feel that this feature should have a reserved option rather then use metadata. On 09/25/14 17:26, Daniel P. Berrange wrote: On Thu, Sep 25, 2014 at 05:23:22PM +0200, Pasquale Porreca wrote: This is correct Daniel, except that that it is done by the virtual firmware/BIOS of the virtual machine and not by the OS (not yet installed at that time). This is the reason we thought about UUID: it is yet used by the iPXE client to be included in Bootstrap Protocol messages, it is taken from the uuid field in libvirt template and the uuid in libvirt is set by OpenStack; the only missing passage is the chance to set the UUID in OpenStack instead to have it randomly generated. Having another user defined tag in libvirt won't help for our issue, since it won't be included in Bootstrap Protocol messages, not without changes in the virtual BIOS/firmware (as you stated too) and honestly my team doesn't have interest in this (neither the competence). I don't think the configdrive or metadata service would help either: the OS on the instance is not yet installed at that time (the target if the network boot is exactly to install the OS on the instance!), so it won't be able to mount it. Ok, yes, if we're considering the DHCP client inside the iPXE BIOS blob, then I don't see any currently viable options besides UUID. There's no mechanism for passing any other data into iPXE that I am aware of, though if there is a desire todo that it could be raised on the QEMU mailing list for discussion. Regards, Daniel ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
On 09/30/2014 06:53 AM, Pasquale Porreca wrote:
Going back to my original question, I would like to know:
1) Is it acceptable to have the UUID passed from client side?
FWIW, Glance has supported supplying the newly-created image's ID in its
API for a long time, and it's never been an issue. On the database side,
you still need to do a primary key lookup to ensure you aren't violating
any constraints, regardless of whether you are doing:
obj.id = uuid.uuid4()
on the controller side or whether you are doing:
req_payload = {
id: uuid.uuid4(),
name: blah
}
client.do_post(jsonutils.dumps(req_payload)..)
on the client side.
I don't really see much of an issue with allowing a user to pass an
opaque identifier for objects on creation.
-jay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
I will briefly explain our use case. This idea is related to another project to enable the network boot in OpenStack https://blueprints.launchpad.net/nova/+spec/pxe-boot-instance We want to make use of the extra-dhcp-opt to indicate as tftp server a specific instance inside our deployed system, so it will provide the right operating system to the other instances booting from network (once the feature from the linked blueprint will be implemented). On the tftp server we want to be able to filter what boot file to provide to different class of instances and our idea was to identify each class with 2 hexadecimal of the UUID (while the rest would be random generated, still granting its uniqueness). Anyway this is a customization for our specific environment and for a feature that is still in early proposal stage, so we wanted to propose as a separate feature to allow user custom UUID and manage the generation out of OpenStack. On 09/24/14 23:15, Matt Riedemann wrote: On 9/24/2014 3:17 PM, Dean Troyer wrote: On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka rpodoly...@mirantis.com mailto:rpodoly...@mirantis.com wrote: Are there any known gotchas with support of this feature in REST APIs (in general)? I'd be worried about relying on a user-defined attribute in that use case, that's ripe for a DOS. Since these are cloud-unique I wouldn't even need to be in your project to block you from creating that clone instance if I knew your UUID. dt -- Dean Troyer dtro...@gmail.com mailto:dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev We talked about this a bit before approving the 'enforce-unique-instance-uuid-in-db' blueprint [1]. As far as we knew there was no one using null instance UUIDs or duplicates for that matter. The instance object already enforces that the UUID field is unique but the database schema doesn't. I'll be re-proposing that for Kilo when it opens up. If it's a matter of tagging an instance, there is also the tags blueprint [2] which will probably be proposed again for Kilo. [1] https://blueprints.launchpad.net/nova/+spec/enforce-unique-instance-uuid-in-db [2] https://blueprints.launchpad.net/nova/+spec/tag-instances -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
On 09/25/2014 04:18 AM, Pasquale Porreca wrote: I will briefly explain our use case. This idea is related to another project to enable the network boot in OpenStack https://blueprints.launchpad.net/nova/+spec/pxe-boot-instance We want to make use of the extra-dhcp-opt to indicate as tftp server a specific instance inside our deployed system, so it will provide the right operating system to the other instances booting from network (once the feature from the linked blueprint will be implemented). On the tftp server we want to be able to filter what boot file to provide to different class of instances and our idea was to identify each class with 2 hexadecimal of the UUID (while the rest would be random generated, still granting its uniqueness). It seems like this would still be achievable using the instance tags feature that Matt mentioned. And it would be more clear since you could use human readable class names rather than relying on knowing that part of the UUID had special meaning. If you have a need to add specific information to an instance like 'boot class' or want to indicate that an instance in two different clouds is actually the same one, the Pumphouse use case, that information should be something we layer on top of an instance and not something we encode in the UUID. Anyway this is a customization for our specific environment and for a feature that is still in early proposal stage, so we wanted to propose as a separate feature to allow user custom UUID and manage the generation out of OpenStack. On 09/24/14 23:15, Matt Riedemann wrote: On 9/24/2014 3:17 PM, Dean Troyer wrote: On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka rpodoly...@mirantis.com mailto:rpodoly...@mirantis.com wrote: Are there any known gotchas with support of this feature in REST APIs (in general)? I'd be worried about relying on a user-defined attribute in that use case, that's ripe for a DOS. Since these are cloud-unique I wouldn't even need to be in your project to block you from creating that clone instance if I knew your UUID. dt -- Dean Troyer dtro...@gmail.com mailto:dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev We talked about this a bit before approving the 'enforce-unique-instance-uuid-in-db' blueprint [1]. As far as we knew there was no one using null instance UUIDs or duplicates for that matter. The instance object already enforces that the UUID field is unique but the database schema doesn't. I'll be re-proposing that for Kilo when it opens up. If it's a matter of tagging an instance, there is also the tags blueprint [2] which will probably be proposed again for Kilo. [1] https://blueprints.launchpad.net/nova/+spec/enforce-unique-instance-uuid-in-db [2] https://blueprints.launchpad.net/nova/+spec/tag-instances ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
The problem to use a different tag than UUID is that it won't be possible (for what I know) to include this tag in the Bootstrap Protocol messages exchanged during the pre-boot phase. Our original idea was to use the Client-identifier (option 61) or Vendor class identifier (option 60) of the dhcp request to achieve our target, but these fields cannot be controlled in libvirt template and so they cannot be set in OpenStack either. Instead the UUID is set it the libvirt template by OpenStack and it is included in the messages exchanged in the pre-boot phase (option 97) by the instance trying to boot from network. Reference: http://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml On 09/25/14 14:43, Andrew Laski wrote: On 09/25/2014 04:18 AM, Pasquale Porreca wrote: I will briefly explain our use case. This idea is related to another project to enable the network boot in OpenStack https://blueprints.launchpad.net/nova/+spec/pxe-boot-instance We want to make use of the extra-dhcp-opt to indicate as tftp server a specific instance inside our deployed system, so it will provide the right operating system to the other instances booting from network (once the feature from the linked blueprint will be implemented). On the tftp server we want to be able to filter what boot file to provide to different class of instances and our idea was to identify each class with 2 hexadecimal of the UUID (while the rest would be random generated, still granting its uniqueness). It seems like this would still be achievable using the instance tags feature that Matt mentioned. And it would be more clear since you could use human readable class names rather than relying on knowing that part of the UUID had special meaning. If you have a need to add specific information to an instance like 'boot class' or want to indicate that an instance in two different clouds is actually the same one, the Pumphouse use case, that information should be something we layer on top of an instance and not something we encode in the UUID. Anyway this is a customization for our specific environment and for a feature that is still in early proposal stage, so we wanted to propose as a separate feature to allow user custom UUID and manage the generation out of OpenStack. On 09/24/14 23:15, Matt Riedemann wrote: On 9/24/2014 3:17 PM, Dean Troyer wrote: On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka rpodoly...@mirantis.com mailto:rpodoly...@mirantis.com wrote: Are there any known gotchas with support of this feature in REST APIs (in general)? I'd be worried about relying on a user-defined attribute in that use case, that's ripe for a DOS. Since these are cloud-unique I wouldn't even need to be in your project to block you from creating that clone instance if I knew your UUID. dt -- Dean Troyer dtro...@gmail.com mailto:dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev We talked about this a bit before approving the 'enforce-unique-instance-uuid-in-db' blueprint [1]. As far as we knew there was no one using null instance UUIDs or duplicates for that matter. The instance object already enforces that the UUID field is unique but the database schema doesn't. I'll be re-proposing that for Kilo when it opens up. If it's a matter of tagging an instance, there is also the tags blueprint [2] which will probably be proposed again for Kilo. [1] https://blueprints.launchpad.net/nova/+spec/enforce-unique-instance-uuid-in-db [2] https://blueprints.launchpad.net/nova/+spec/tag-instances ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
On 9/25/2014 8:26 AM, Pasquale Porreca wrote: The problem to use a different tag than UUID is that it won't be possible (for what I know) to include this tag in the Bootstrap Protocol messages exchanged during the pre-boot phase. Our original idea was to use the Client-identifier (option 61) or Vendor class identifier (option 60) of the dhcp request to achieve our target, but these fields cannot be controlled in libvirt template and so they cannot be set in OpenStack either. Instead the UUID is set it the libvirt template by OpenStack and it is included in the messages exchanged in the pre-boot phase (option 97) by the instance trying to boot from network. Reference: http://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml On 09/25/14 14:43, Andrew Laski wrote: On 09/25/2014 04:18 AM, Pasquale Porreca wrote: I will briefly explain our use case. This idea is related to another project to enable the network boot in OpenStack https://blueprints.launchpad.net/nova/+spec/pxe-boot-instance We want to make use of the extra-dhcp-opt to indicate as tftp server a specific instance inside our deployed system, so it will provide the right operating system to the other instances booting from network (once the feature from the linked blueprint will be implemented). On the tftp server we want to be able to filter what boot file to provide to different class of instances and our idea was to identify each class with 2 hexadecimal of the UUID (while the rest would be random generated, still granting its uniqueness). It seems like this would still be achievable using the instance tags feature that Matt mentioned. And it would be more clear since you could use human readable class names rather than relying on knowing that part of the UUID had special meaning. If you have a need to add specific information to an instance like 'boot class' or want to indicate that an instance in two different clouds is actually the same one, the Pumphouse use case, that information should be something we layer on top of an instance and not something we encode in the UUID. Anyway this is a customization for our specific environment and for a feature that is still in early proposal stage, so we wanted to propose as a separate feature to allow user custom UUID and manage the generation out of OpenStack. On 09/24/14 23:15, Matt Riedemann wrote: On 9/24/2014 3:17 PM, Dean Troyer wrote: On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka rpodoly...@mirantis.com mailto:rpodoly...@mirantis.com wrote: Are there any known gotchas with support of this feature in REST APIs (in general)? I'd be worried about relying on a user-defined attribute in that use case, that's ripe for a DOS. Since these are cloud-unique I wouldn't even need to be in your project to block you from creating that clone instance if I knew your UUID. dt -- Dean Troyer dtro...@gmail.com mailto:dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev We talked about this a bit before approving the 'enforce-unique-instance-uuid-in-db' blueprint [1]. As far as we knew there was no one using null instance UUIDs or duplicates for that matter. The instance object already enforces that the UUID field is unique but the database schema doesn't. I'll be re-proposing that for Kilo when it opens up. If it's a matter of tagging an instance, there is also the tags blueprint [2] which will probably be proposed again for Kilo. [1] https://blueprints.launchpad.net/nova/+spec/enforce-unique-instance-uuid-in-db [2] https://blueprints.launchpad.net/nova/+spec/tag-instances ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev If it's a matter of getting the instance tag information down to the libvirt driver on boot that shouldn't be a problem, there are others asking for similar things, i.e. I want to tag my instances at create time and store that tag metadata in some namespace in the libvirt domain xml so I can have an application outside of openstack consuming those domain xml's and reading that custom namespace information. -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
On Thu, Sep 25, 2014 at 09:19:03AM -0500, Matt Riedemann wrote: On 9/25/2014 8:26 AM, Pasquale Porreca wrote: The problem to use a different tag than UUID is that it won't be possible (for what I know) to include this tag in the Bootstrap Protocol messages exchanged during the pre-boot phase. Our original idea was to use the Client-identifier (option 61) or Vendor class identifier (option 60) of the dhcp request to achieve our target, but these fields cannot be controlled in libvirt template and so they cannot be set in OpenStack either. Instead the UUID is set it the libvirt template by OpenStack and it is included in the messages exchanged in the pre-boot phase (option 97) by the instance trying to boot from network. Reference: http://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml [snip] If it's a matter of getting the instance tag information down to the libvirt driver on boot that shouldn't be a problem, there are others asking for similar things, i.e. I want to tag my instances at create time and store that tag metadata in some namespace in the libvirt domain xml so I can have an application outside of openstack consuming those domain xml's and reading that custom namespace information. Perhaps I'm misunderstanding something, but isn't the DHCP client that needs to send the tag running in the guest OS ? Libvirt is involved wrt UUID, because UUID is populated in the guest's virtual BIOS and then extracted by the guest OS and from there used by the DHCP client. If we're talking about making a different tag/identifier available for the DHCP client, then this is probably not going to involve libvirt unless it also gets pushed up via the virtual BIOS. IOW, couldn't you just pass whatever tag is needed to the guest OS via the configdrive or metadata service. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
This is correct Daniel, except that that it is done by the virtual firmware/BIOS of the virtual machine and not by the OS (not yet installed at that time). This is the reason we thought about UUID: it is yet used by the iPXE client to be included in Bootstrap Protocol messages, it is taken from the uuid field in libvirt template and the uuid in libvirt is set by OpenStack; the only missing passage is the chance to set the UUID in OpenStack instead to have it randomly generated. Having another user defined tag in libvirt won't help for our issue, since it won't be included in Bootstrap Protocol messages, not without changes in the virtual BIOS/firmware (as you stated too) and honestly my team doesn't have interest in this (neither the competence). I don't think the configdrive or metadata service would help either: the OS on the instance is not yet installed at that time (the target if the network boot is exactly to install the OS on the instance!), so it won't be able to mount it. On 09/25/14 16:24, Daniel P. Berrange wrote: On Thu, Sep 25, 2014 at 09:19:03AM -0500, Matt Riedemann wrote: On 9/25/2014 8:26 AM, Pasquale Porreca wrote: The problem to use a different tag than UUID is that it won't be possible (for what I know) to include this tag in the Bootstrap Protocol messages exchanged during the pre-boot phase. Our original idea was to use the Client-identifier (option 61) or Vendor class identifier (option 60) of the dhcp request to achieve our target, but these fields cannot be controlled in libvirt template and so they cannot be set in OpenStack either. Instead the UUID is set it the libvirt template by OpenStack and it is included in the messages exchanged in the pre-boot phase (option 97) by the instance trying to boot from network. Reference: http://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml [snip] If it's a matter of getting the instance tag information down to the libvirt driver on boot that shouldn't be a problem, there are others asking for similar things, i.e. I want to tag my instances at create time and store that tag metadata in some namespace in the libvirt domain xml so I can have an application outside of openstack consuming those domain xml's and reading that custom namespace information. Perhaps I'm misunderstanding something, but isn't the DHCP client that needs to send the tag running in the guest OS ? Libvirt is involved wrt UUID, because UUID is populated in the guest's virtual BIOS and then extracted by the guest OS and from there used by the DHCP client. If we're talking about making a different tag/identifier available for the DHCP client, then this is probably not going to involve libvirt unless it also gets pushed up via the virtual BIOS. IOW, couldn't you just pass whatever tag is needed to the guest OS via the configdrive or metadata service. Regards, Daniel -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
On Thu, Sep 25, 2014 at 05:23:22PM +0200, Pasquale Porreca wrote: This is correct Daniel, except that that it is done by the virtual firmware/BIOS of the virtual machine and not by the OS (not yet installed at that time). This is the reason we thought about UUID: it is yet used by the iPXE client to be included in Bootstrap Protocol messages, it is taken from the uuid field in libvirt template and the uuid in libvirt is set by OpenStack; the only missing passage is the chance to set the UUID in OpenStack instead to have it randomly generated. Having another user defined tag in libvirt won't help for our issue, since it won't be included in Bootstrap Protocol messages, not without changes in the virtual BIOS/firmware (as you stated too) and honestly my team doesn't have interest in this (neither the competence). I don't think the configdrive or metadata service would help either: the OS on the instance is not yet installed at that time (the target if the network boot is exactly to install the OS on the instance!), so it won't be able to mount it. Ok, yes, if we're considering the DHCP client inside the iPXE BIOS blob, then I don't see any currently viable options besides UUID. There's no mechanism for passing any other data into iPXE that I am aware of, though if there is a desire todo that it could be raised on the QEMU mailing list for discussion. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Create an instance with a custom uuid
Hello I would like to be able to specify the UUID of an instance when I create it. I found this discussion about this matter: https://lists.launchpad.net/openstack/msg22387.html but I could not find any blueprint, anyway I understood this modification should not comport any particular issue. Would it be acceptable to pass the uuid as metadata, or should I instead modify the api if I want to set the UUID from the novaclient? Best regards -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
+1. Or at least provide a way to specify an external UUID for the instance, and can retrieve the instance through the external UUID which may be linked to external system's object. Chaoyi Huang ( joehuang ) 发件人: Pasquale Porreca [pasquale.porr...@dektech.com.au] 发送时间: 2014年9月24日 21:08 收件人: openstack-dev@lists.openstack.org 主题: [openstack-dev] [nova] Create an instance with a custom uuid Hello I would like to be able to specify the UUID of an instance when I create it. I found this discussion about this matter: https://lists.launchpad.net/openstack/msg22387.html but I could not find any blueprint, anyway I understood this modification should not comport any particular issue. Would it be acceptable to pass the uuid as metadata, or should I instead modify the api if I want to set the UUID from the novaclient? Best regards -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
Hi Joe, Tools like Pumphouse [1] (migrates workloads, e.g. instances, between two OpenStack clouds) would benefit from supporting this (Pumphouse would be able to replicate user instances in a new cloud up to their UUIDs). Are there any known gotchas with support of this feature in REST APIs (in general)? Thanks, Roman [1] https://github.com/MirantisLabs/pumphouse On Wed, Sep 24, 2014 at 10:23 AM, Joe Gordon joe.gord...@gmail.com wrote: Whats the use case for this? We should be thorough when making API changes like this. On Wed, Sep 24, 2014 at 6:56 AM, joehuang joehu...@huawei.com wrote: +1. Or at least provide a way to specify an external UUID for the instance, and can retrieve the instance through the external UUID which may be linked to external system's object. Chaoyi Huang ( joehuang ) 发件人: Pasquale Porreca [pasquale.porr...@dektech.com.au] 发送时间: 2014年9月24日 21:08 收件人: openstack-dev@lists.openstack.org 主题: [openstack-dev] [nova] Create an instance with a custom uuid Hello I would like to be able to specify the UUID of an instance when I create it. I found this discussion about this matter: https://lists.launchpad.net/openstack/msg22387.html but I could not find any blueprint, anyway I understood this modification should not comport any particular issue. Would it be acceptable to pass the uuid as metadata, or should I instead modify the api if I want to set the UUID from the novaclient? Best regards -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka rpodoly...@mirantis.com wrote: Are there any known gotchas with support of this feature in REST APIs (in general)? I'd be worried about relying on a user-defined attribute in that use case, that's ripe for a DOS. Since these are cloud-unique I wouldn't even need to be in your project to block you from creating that clone instance if I knew your UUID. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
On 9/24/2014 3:17 PM, Dean Troyer wrote: On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka rpodoly...@mirantis.com mailto:rpodoly...@mirantis.com wrote: Are there any known gotchas with support of this feature in REST APIs (in general)? I'd be worried about relying on a user-defined attribute in that use case, that's ripe for a DOS. Since these are cloud-unique I wouldn't even need to be in your project to block you from creating that clone instance if I knew your UUID. dt -- Dean Troyer dtro...@gmail.com mailto:dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev We talked about this a bit before approving the 'enforce-unique-instance-uuid-in-db' blueprint [1]. As far as we knew there was no one using null instance UUIDs or duplicates for that matter. The instance object already enforces that the UUID field is unique but the database schema doesn't. I'll be re-proposing that for Kilo when it opens up. If it's a matter of tagging an instance, there is also the tags blueprint [2] which will probably be proposed again for Kilo. [1] https://blueprints.launchpad.net/nova/+spec/enforce-unique-instance-uuid-in-db [2] https://blueprints.launchpad.net/nova/+spec/tag-instances -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Create an instance with a custom uuid
It can also be done by external system to do the mapping of UUID, but not to change the API. When I develop Windows app, for example CListBox, for each list item, you can set item-data(four bytes). Sometimes I use the item-data to store the index or pointer of the object the item belongs to. If the user click the list box, and select one item, the regarding object could be find very fast. This is also not good idea for we have to change the API and “that's ripe for a DOS“ (rpodoly...@mirantis.commailto:rpodoly...@mirantis.com). So I admit that I give +1 too early. Thanks to Joe and rpodolyaka. Best Regards Chaoyi Huang ( Joe Huang ) 发件人: Joe Gordon [mailto:joe.gord...@gmail.com] 发送时间: 2014年9月25日 1:23 收件人: OpenStack Development Mailing List (not for usage questions) 主题: Re: [openstack-dev] [nova] Create an instance with a custom uuid Whats the use case for this? We should be thorough when making API changes like this. On Wed, Sep 24, 2014 at 6:56 AM, joehuang joehu...@huawei.commailto:joehu...@huawei.com wrote: +1. Or at least provide a way to specify an external UUID for the instance, and can retrieve the instance through the external UUID which may be linked to external system's object. Chaoyi Huang ( joehuang ) 发件人: Pasquale Porreca [pasquale.porr...@dektech.com.aumailto:pasquale.porr...@dektech.com.au] 发送时间: 2014年9月24日 21:08 收件人: openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org 主题: [openstack-dev] [nova] Create an instance with a custom uuid Hello I would like to be able to specify the UUID of an instance when I create it. I found this discussion about this matter: https://lists.launchpad.net/openstack/msg22387.html but I could not find any blueprint, anyway I understood this modification should not comport any particular issue. Would it be acceptable to pass the uuid as metadata, or should I instead modify the api if I want to set the UUID from the novaclient? Best regards -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805tel:%2B39%203394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
