Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-10-05 Thread Angelo Matarazzo

Hi all,
while I was working on blueprint Pasquale addressed this issue.
In the use case explained  both features are involved
but the user defined UUID does not depend strictly on original blueprint 
but it can be a new one.
If there is no objection I could write a blueprint/specification 
regarding to last solution.

Any thought?

BR,
Angelo

On 01/10/2014 18:07, Joe Gordon wrote:



On Wed, Oct 1, 2014 at 8:29 AM, Solly Ross sr...@redhat.com 
mailto:sr...@redhat.com wrote:


(response inline)

- Original Message -
 From: Pasquale Porreca pasquale.porr...@dektech.com.au
mailto:pasquale.porr...@dektech.com.au
 To: openstack-dev@lists.openstack.org
mailto:openstack-dev@lists.openstack.org
 Sent: Wednesday, October 1, 2014 11:08:50 AM
 Subject: Re: [openstack-dev] [nova] Create an instance with a
custom uuid

 Thank you for the answers.

 I understood the concerns about having the UUID completely user
defined and I
 also understand Nova has no interest in supporting a customized
algorithm to
 generate UUID. Anyway I may have found a solution that will
cover my use
 case and respect the standard for UUID (RFC 4122
 http://www.ietf.org/rfc/rfc4122.txt ) .

 The generation of the UUID in Nova make use of the function
uuid4() from the
 module uuid.py to have an UUID (pseudo)random, according to
version 4
 described in RFC 4122. Anyway this is not the only algorithm
supported in
 the standard (and implemented yet in uuid.py ).

 In particular I focused my attention on UUID version 1 and the
method
 uuid1(node=None, clock_seq=None) that allows to pass as
parameter a part of
 the UUID ( node is the field containing the last 12 hexadecimal
digits of
 the UUID).

 So my idea was to give the chance to the user to set uiid
version (1 or 4,
 with the latter as default) when creating a new instance and in
case of
 version 1 to pass optionally a value for parameter node .

I would think that we could just have a node parameter here, and
automatically
use version 1 if that parameter is passed (if we decided to go the
route
of changing the current UUID behavior).


From what I gather this requested change in API is based on for your 
blueprint 
https://blueprints.launchpad.net/nova/+spec/pxe-boot-instance. Since 
your blueprint is not approved yet discussing further work to improve 
it is a bit premature.




 Any thoughts?

 On 09/30/14 14:07, Andrew Laski wrote:



 On 09/30/2014 06:53 AM, Pasquale Porreca wrote:


 Going back to my original question, I would like to know:

 1) Is it acceptable to have the UUID passed from client side?

 In my opinion, no. This opens a door to issues we currently
don't need to
 deal with, and use cases I don't think Nova should support. Another
 possibility, which I don't like either, would be to pass in some
data which
 could influence the generation of the UUID to satisfy requirements.

 But there was a suggestion to look into addressing your use case
on the QEMU
 mailing list, which I think would be a better approach.




 2) What is the correct way to do it? I started to implement this
feature,
 simply passing it as metadata with key uuid, but I feel that
this feature
 should have a reserved option rather then use metadata.


 On 09/25/14 17:26, Daniel P. Berrange wrote:


 On Thu, Sep 25, 2014 at 05:23:22PM +0200, Pasquale Porreca wrote:


 This is correct Daniel, except that that it is done by the virtual
 firmware/BIOS of the virtual machine and not by the OS (not yet
installed at
 that time).

 This is the reason we thought about UUID: it is yet used by the
iPXE client
 to be included in Bootstrap Protocol messages, it is taken from
the uuid
 field in libvirt template and the uuid in libvirt is set by
OpenStack; the
 only missing passage is the chance to set the UUID in OpenStack
instead to
 have it randomly generated.

 Having another user defined tag in libvirt won't help for our
issue, since
 it won't be included in Bootstrap Protocol messages, not without
changes in
 the virtual BIOS/firmware (as you stated too) and honestly my
team doesn't
 have interest in this (neither the competence).

 I don't think the configdrive or metadata service would help
either: the OS
 on the instance is not yet installed at that time (the target if
the network
 boot is exactly to install the OS on the instance!), so it won't
be able to
 mount it.
 Ok, yes, if we're considering the DHCP client inside the iPXE BIOS
 blob, then I don't see any currently viable options besides UUID.
 There's no mechanism

Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-10-01 Thread Pasquale Porreca

Thank you for the answers.

I understood the concerns about having the UUID completely user defined 
and I also understand Nova has no interest in supporting a customized 
algorithm to generate UUID. Anyway I may have found a solution that will 
cover my use case and respect the standard for UUID (RFC 4122 
http://www.ietf.org/rfc/rfc4122.txt) .


The generation of the UUID in Nova make use of the function /uuid4()/ 
from the module /uuid.py/ to have an UUID (pseudo)random, according to 
version 4 described in RFC 4122. Anyway this is not the only algorithm 
supported in the standard (and implemented yet in /uuid.py/).


In particular I focused my attention on UUID version 1 and the method 
/uuid1(node=None, clock_seq=None)/ that allows to pass as parameter a 
part of the UUID (/node/ is the field containing the last 12 hexadecimal 
digits of the UUID).


So my idea was to give the chance to the user to set uiid version (1 or 
4, with the latter as default) when creating a new instance and in case 
of version 1  to pass optionally a value for parameter /node/.


Any thoughts?

On 09/30/14 14:07, Andrew Laski wrote:


On 09/30/2014 06:53 AM, Pasquale Porreca wrote:

Going back to my original question, I would like to know:

1) Is it acceptable to have the UUID passed from client side?


In my opinion, no.  This opens a door to issues we currently don't 
need to deal with, and use cases I don't think Nova should support. 
Another possibility, which I don't like either, would be to pass in 
some data which could influence the generation of the UUID to satisfy 
requirements.


But there was a suggestion to look into addressing your use case on 
the QEMU mailing list, which I think would be a better approach.




2) What is the correct way to do it? I started to implement this 
feature, simply passing it as metadata with key uuid, but I feel that 
this feature should have a reserved option rather then use metadata.



On 09/25/14 17:26, Daniel P. Berrange wrote:

On Thu, Sep 25, 2014 at 05:23:22PM +0200, Pasquale Porreca wrote:

This is correct Daniel, except that that it is done by the virtual
firmware/BIOS of the virtual machine and not by the OS (not yet 
installed at

that time).

This is the reason we thought about UUID: it is yet used by the 
iPXE client
to be included in Bootstrap Protocol messages, it is taken from the 
uuid
field in libvirt template and the uuid in libvirt is set by 
OpenStack; the
only missing passage is the chance to set the UUID in OpenStack 
instead to

have it randomly generated.

Having another user defined tag in libvirt won't help for our 
issue, since
it won't be included in Bootstrap Protocol messages, not without 
changes in
the virtual BIOS/firmware (as you stated too) and honestly my team 
doesn't

have interest in this (neither the competence).

I don't think the configdrive or metadata service would help 
either: the OS
on the instance is not yet installed at that time (the target if 
the network
boot is exactly to install the OS on the instance!), so it won't be 
able to

mount it.

Ok, yes, if we're considering the DHCP client inside the iPXE BIOS
blob, then I don't see any currently viable options besides UUID.
There's no mechanism for passing any other data into iPXE that I
am aware of, though if there is a desire todo that it could be
raised on the QEMU mailing list for discussion.


Regards,
Daniel





___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
Pasquale Porreca

DEK Technologies
Via dei Castelli Romani, 22
00040 Pomezia (Roma)

Mobile +39 3394823805
Skype paskporr

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-10-01 Thread Joe Gordon
On Wed, Oct 1, 2014 at 8:29 AM, Solly Ross sr...@redhat.com wrote:

 (response inline)

 - Original Message -
  From: Pasquale Porreca pasquale.porr...@dektech.com.au
  To: openstack-dev@lists.openstack.org
  Sent: Wednesday, October 1, 2014 11:08:50 AM
  Subject: Re: [openstack-dev] [nova] Create an instance with a custom uuid
 
  Thank you for the answers.
 
  I understood the concerns about having the UUID completely user defined
 and I
  also understand Nova has no interest in supporting a customized
 algorithm to
  generate UUID. Anyway I may have found a solution that will cover my use
  case and respect the standard for UUID (RFC 4122
  http://www.ietf.org/rfc/rfc4122.txt ) .
 
  The generation of the UUID in Nova make use of the function uuid4() from
 the
  module uuid.py to have an UUID (pseudo)random, according to version 4
  described in RFC 4122. Anyway this is not the only algorithm supported in
  the standard (and implemented yet in uuid.py ).
 
  In particular I focused my attention on UUID version 1 and the method
  uuid1(node=None, clock_seq=None) that allows to pass as parameter a part
 of
  the UUID ( node is the field containing the last 12 hexadecimal digits of
  the UUID).
 
  So my idea was to give the chance to the user to set uiid version (1 or
 4,
  with the latter as default) when creating a new instance and in case of
  version 1 to pass optionally a value for parameter node .

 I would think that we could just have a node parameter here, and
 automatically
 use version 1 if that parameter is passed (if we decided to go the route
 of changing the current UUID behavior).


From what I gather this requested change in API is based on for your
blueprint https://blueprints.launchpad.net/nova/+spec/pxe-boot-instance.
Since your blueprint is not approved yet discussing further work to improve
it is a bit premature.



 
  Any thoughts?
 
  On 09/30/14 14:07, Andrew Laski wrote:
 
 
 
  On 09/30/2014 06:53 AM, Pasquale Porreca wrote:
 
 
  Going back to my original question, I would like to know:
 
  1) Is it acceptable to have the UUID passed from client side?
 
  In my opinion, no. This opens a door to issues we currently don't need to
  deal with, and use cases I don't think Nova should support. Another
  possibility, which I don't like either, would be to pass in some data
 which
  could influence the generation of the UUID to satisfy requirements.
 
  But there was a suggestion to look into addressing your use case on the
 QEMU
  mailing list, which I think would be a better approach.
 
 
 
 
  2) What is the correct way to do it? I started to implement this feature,
  simply passing it as metadata with key uuid, but I feel that this feature
  should have a reserved option rather then use metadata.
 
 
  On 09/25/14 17:26, Daniel P. Berrange wrote:
 
 
  On Thu, Sep 25, 2014 at 05:23:22PM +0200, Pasquale Porreca wrote:
 
 
  This is correct Daniel, except that that it is done by the virtual
  firmware/BIOS of the virtual machine and not by the OS (not yet
 installed at
  that time).
 
  This is the reason we thought about UUID: it is yet used by the iPXE
 client
  to be included in Bootstrap Protocol messages, it is taken from the
 uuid
  field in libvirt template and the uuid in libvirt is set by OpenStack;
 the
  only missing passage is the chance to set the UUID in OpenStack instead
 to
  have it randomly generated.
 
  Having another user defined tag in libvirt won't help for our issue,
 since
  it won't be included in Bootstrap Protocol messages, not without changes
 in
  the virtual BIOS/firmware (as you stated too) and honestly my team
 doesn't
  have interest in this (neither the competence).
 
  I don't think the configdrive or metadata service would help either: the
 OS
  on the instance is not yet installed at that time (the target if the
 network
  boot is exactly to install the OS on the instance!), so it won't be able
 to
  mount it.
  Ok, yes, if we're considering the DHCP client inside the iPXE BIOS
  blob, then I don't see any currently viable options besides UUID.
  There's no mechanism for passing any other data into iPXE that I
  am aware of, though if there is a desire todo that it could be
  raised on the QEMU mailing list for discussion.
 
 
  Regards,
  Daniel
 
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
  --
  Pasquale Porreca
 
  DEK Technologies
  Via dei Castelli Romani, 22
  00040 Pomezia (Roma)
 
  Mobile +39 3394823805
  Skype paskporr
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

 Best Regards,
 Solly Ross

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin

Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-30 Thread Pasquale Porreca

Going back to my original question, I would like to know:

1) Is it acceptable to have the UUID passed from client side?

2) What is the correct way to do it? I started to implement this 
feature, simply passing it as metadata with key uuid, but I feel that 
this feature should have a reserved option rather then use metadata.



On 09/25/14 17:26, Daniel P. Berrange wrote:

On Thu, Sep 25, 2014 at 05:23:22PM +0200, Pasquale Porreca wrote:

This is correct Daniel, except that that it is done by the virtual
firmware/BIOS of the virtual machine and not by the OS (not yet installed at
that time).

This is the reason we thought about UUID: it is yet used by the iPXE client
to be included in Bootstrap Protocol messages, it is taken from the uuid
field in libvirt template and the uuid in libvirt is set by OpenStack; the
only missing passage is the chance to set the UUID in OpenStack instead to
have it randomly generated.

Having another user defined tag in libvirt won't help for our issue, since
it won't be included in Bootstrap Protocol messages, not without changes in
the virtual BIOS/firmware (as you stated too) and honestly my team doesn't
have interest in this (neither the competence).

I don't think the configdrive or metadata service would help either: the OS
on the instance is not yet installed at that time (the target if the network
boot is exactly to install the OS on the instance!), so it won't be able to
mount it.

Ok, yes, if we're considering the DHCP client inside the iPXE BIOS
blob, then I don't see any currently viable options besides UUID.
There's no mechanism for passing any other data into iPXE that I
am aware of, though if there is a desire todo that it could be
raised on the QEMU mailing list for discussion.


Regards,
Daniel


--
Pasquale Porreca

DEK Technologies
Via dei Castelli Romani, 22
00040 Pomezia (Roma)

Mobile +39 3394823805
Skype paskporr


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-30 Thread Andrew Laski


On 09/30/2014 06:53 AM, Pasquale Porreca wrote:

Going back to my original question, I would like to know:

1) Is it acceptable to have the UUID passed from client side?


In my opinion, no.  This opens a door to issues we currently don't need 
to deal with, and use cases I don't think Nova should support. Another 
possibility, which I don't like either, would be to pass in some data 
which could influence the generation of the UUID to satisfy requirements.


But there was a suggestion to look into addressing your use case on the 
QEMU mailing list, which I think would be a better approach.




2) What is the correct way to do it? I started to implement this 
feature, simply passing it as metadata with key uuid, but I feel that 
this feature should have a reserved option rather then use metadata.



On 09/25/14 17:26, Daniel P. Berrange wrote:

On Thu, Sep 25, 2014 at 05:23:22PM +0200, Pasquale Porreca wrote:

This is correct Daniel, except that that it is done by the virtual
firmware/BIOS of the virtual machine and not by the OS (not yet 
installed at

that time).

This is the reason we thought about UUID: it is yet used by the iPXE 
client
to be included in Bootstrap Protocol messages, it is taken from the 
uuid
field in libvirt template and the uuid in libvirt is set by 
OpenStack; the
only missing passage is the chance to set the UUID in OpenStack 
instead to

have it randomly generated.

Having another user defined tag in libvirt won't help for our issue, 
since
it won't be included in Bootstrap Protocol messages, not without 
changes in
the virtual BIOS/firmware (as you stated too) and honestly my team 
doesn't

have interest in this (neither the competence).

I don't think the configdrive or metadata service would help either: 
the OS
on the instance is not yet installed at that time (the target if the 
network
boot is exactly to install the OS on the instance!), so it won't be 
able to

mount it.

Ok, yes, if we're considering the DHCP client inside the iPXE BIOS
blob, then I don't see any currently viable options besides UUID.
There's no mechanism for passing any other data into iPXE that I
am aware of, though if there is a desire todo that it could be
raised on the QEMU mailing list for discussion.


Regards,
Daniel





___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-30 Thread Jay Pipes

On 09/30/2014 06:53 AM, Pasquale Porreca wrote:

Going back to my original question, I would like to know:

1) Is it acceptable to have the UUID passed from client side?


FWIW, Glance has supported supplying the newly-created image's ID in its 
API for a long time, and it's never been an issue. On the database side, 
you still need to do a primary key lookup to ensure you aren't violating 
any constraints, regardless of whether you are doing:


 obj.id = uuid.uuid4()

on the controller side or whether you are doing:

 req_payload = {
   id: uuid.uuid4(),
   name: blah
 }
 client.do_post(jsonutils.dumps(req_payload)..)

on the client side.

I don't really see much of an issue with allowing a user to pass an 
opaque identifier for objects on creation.


-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-25 Thread Pasquale Porreca
I will briefly explain our use case. This idea is related to another 
project to enable the network boot in OpenStack 
https://blueprints.launchpad.net/nova/+spec/pxe-boot-instance


We want to make use of the extra-dhcp-opt to indicate as tftp server a 
specific instance inside our deployed system, so it will provide the 
right operating system to the other instances booting from network (once 
the feature from the linked blueprint will be implemented).


On the tftp server we want to be able to filter what boot file to 
provide to different class of instances and our idea was to identify 
each class with 2 hexadecimal of the UUID (while the rest would be 
random generated, still granting its uniqueness).


Anyway this is a customization for our specific environment and for a 
feature that is still in early proposal stage, so we wanted to propose 
as a separate feature to allow user custom UUID and manage the 
generation out of OpenStack.



On 09/24/14 23:15, Matt Riedemann wrote:



On 9/24/2014 3:17 PM, Dean Troyer wrote:

On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka
rpodoly...@mirantis.com mailto:rpodoly...@mirantis.com wrote:

Are there any known gotchas with support of this feature in REST 
APIs

(in general)?


I'd be worried about relying on a user-defined attribute in that use
case, that's ripe for a DOS.  Since these are cloud-unique I wouldn't
even need to be in your project to block you from creating that clone
instance if I knew your UUID.

dt

--

Dean Troyer
dtro...@gmail.com mailto:dtro...@gmail.com


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



We talked about this a bit before approving the 
'enforce-unique-instance-uuid-in-db' blueprint [1].  As far as we knew 
there was no one using null instance UUIDs or duplicates for that matter.


The instance object already enforces that the UUID field is unique but 
the database schema doesn't.  I'll be re-proposing that for Kilo when 
it opens up.


If it's a matter of tagging an instance, there is also the tags 
blueprint [2] which will probably be proposed again for Kilo.


[1] 
https://blueprints.launchpad.net/nova/+spec/enforce-unique-instance-uuid-in-db

[2] https://blueprints.launchpad.net/nova/+spec/tag-instances



--
Pasquale Porreca

DEK Technologies
Via dei Castelli Romani, 22
00040 Pomezia (Roma)

Mobile +39 3394823805
Skype paskporr


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-25 Thread Andrew Laski


On 09/25/2014 04:18 AM, Pasquale Porreca wrote:
I will briefly explain our use case. This idea is related to another 
project to enable the network boot in OpenStack 
https://blueprints.launchpad.net/nova/+spec/pxe-boot-instance


We want to make use of the extra-dhcp-opt to indicate as tftp server a 
specific instance inside our deployed system, so it will provide the 
right operating system to the other instances booting from network 
(once the feature from the linked blueprint will be implemented).


On the tftp server we want to be able to filter what boot file to 
provide to different class of instances and our idea was to identify 
each class with 2 hexadecimal of the UUID (while the rest would be 
random generated, still granting its uniqueness).


It seems like this would still be achievable using the instance tags 
feature that Matt mentioned.  And it would be more clear since you could 
use human readable class names rather than relying on knowing that part 
of the UUID had special meaning.


If you have a need to add specific information to an instance like 'boot 
class' or want to indicate that an instance in two different clouds is 
actually the same one, the Pumphouse use case, that information should 
be something we layer on top of an instance and not something we encode 
in the UUID.



Anyway this is a customization for our specific environment and for a 
feature that is still in early proposal stage, so we wanted to propose 
as a separate feature to allow user custom UUID and manage the 
generation out of OpenStack.

On 09/24/14 23:15, Matt Riedemann wrote:



On 9/24/2014 3:17 PM, Dean Troyer wrote:

On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka
rpodoly...@mirantis.com mailto:rpodoly...@mirantis.com wrote:

Are there any known gotchas with support of this feature in REST 
APIs

(in general)?


I'd be worried about relying on a user-defined attribute in that use
case, that's ripe for a DOS.  Since these are cloud-unique I wouldn't
even need to be in your project to block you from creating that clone
instance if I knew your UUID.

dt

--

Dean Troyer
dtro...@gmail.com mailto:dtro...@gmail.com


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



We talked about this a bit before approving the 
'enforce-unique-instance-uuid-in-db' blueprint [1].  As far as we 
knew there was no one using null instance UUIDs or duplicates for 
that matter.


The instance object already enforces that the UUID field is unique 
but the database schema doesn't.  I'll be re-proposing that for Kilo 
when it opens up.


If it's a matter of tagging an instance, there is also the tags 
blueprint [2] which will probably be proposed again for Kilo.


[1] 
https://blueprints.launchpad.net/nova/+spec/enforce-unique-instance-uuid-in-db

[2] https://blueprints.launchpad.net/nova/+spec/tag-instances






___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-25 Thread Pasquale Porreca
The problem to use a different tag than UUID is that it won't be 
possible (for what I know) to include this tag in the Bootstrap Protocol 
messages exchanged during the pre-boot phase.


Our original idea was to use the Client-identifier (option 61) or Vendor 
class identifier (option 60) of the dhcp request to achieve our target, 
but these fields cannot be controlled in libvirt template and so they 
cannot be set in OpenStack either. Instead the UUID is set it the 
libvirt template by OpenStack and it is included in the messages 
exchanged in the pre-boot phase (option 97) by the instance trying to 
boot from network.


Reference: 
http://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml



On 09/25/14 14:43, Andrew Laski wrote:


On 09/25/2014 04:18 AM, Pasquale Porreca wrote:
I will briefly explain our use case. This idea is related to another 
project to enable the network boot in OpenStack 
https://blueprints.launchpad.net/nova/+spec/pxe-boot-instance


We want to make use of the extra-dhcp-opt to indicate as tftp server 
a specific instance inside our deployed system, so it will provide 
the right operating system to the other instances booting from 
network (once the feature from the linked blueprint will be 
implemented).


On the tftp server we want to be able to filter what boot file to 
provide to different class of instances and our idea was to identify 
each class with 2 hexadecimal of the UUID (while the rest would be 
random generated, still granting its uniqueness).


It seems like this would still be achievable using the instance tags 
feature that Matt mentioned.  And it would be more clear since you 
could use human readable class names rather than relying on knowing 
that part of the UUID had special meaning.


If you have a need to add specific information to an instance like 
'boot class' or want to indicate that an instance in two different 
clouds is actually the same one, the Pumphouse use case, that 
information should be something we layer on top of an instance and not 
something we encode in the UUID.



Anyway this is a customization for our specific environment and for a 
feature that is still in early proposal stage, so we wanted to 
propose as a separate feature to allow user custom UUID and manage 
the generation out of OpenStack.

On 09/24/14 23:15, Matt Riedemann wrote:



On 9/24/2014 3:17 PM, Dean Troyer wrote:

On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka
rpodoly...@mirantis.com mailto:rpodoly...@mirantis.com wrote:

Are there any known gotchas with support of this feature in 
REST APIs

(in general)?


I'd be worried about relying on a user-defined attribute in that use
case, that's ripe for a DOS.  Since these are cloud-unique I wouldn't
even need to be in your project to block you from creating that clone
instance if I knew your UUID.

dt

--

Dean Troyer
dtro...@gmail.com mailto:dtro...@gmail.com


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



We talked about this a bit before approving the 
'enforce-unique-instance-uuid-in-db' blueprint [1].  As far as we 
knew there was no one using null instance UUIDs or duplicates for 
that matter.


The instance object already enforces that the UUID field is unique 
but the database schema doesn't.  I'll be re-proposing that for Kilo 
when it opens up.


If it's a matter of tagging an instance, there is also the tags 
blueprint [2] which will probably be proposed again for Kilo.


[1] 
https://blueprints.launchpad.net/nova/+spec/enforce-unique-instance-uuid-in-db

[2] https://blueprints.launchpad.net/nova/+spec/tag-instances






___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
Pasquale Porreca

DEK Technologies
Via dei Castelli Romani, 22
00040 Pomezia (Roma)

Mobile +39 3394823805
Skype paskporr


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-25 Thread Matt Riedemann



On 9/25/2014 8:26 AM, Pasquale Porreca wrote:

The problem to use a different tag than UUID is that it won't be
possible (for what I know) to include this tag in the Bootstrap Protocol
messages exchanged during the pre-boot phase.

Our original idea was to use the Client-identifier (option 61) or Vendor
class identifier (option 60) of the dhcp request to achieve our target,
but these fields cannot be controlled in libvirt template and so they
cannot be set in OpenStack either. Instead the UUID is set it the
libvirt template by OpenStack and it is included in the messages
exchanged in the pre-boot phase (option 97) by the instance trying to
boot from network.

Reference:
http://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml



On 09/25/14 14:43, Andrew Laski wrote:


On 09/25/2014 04:18 AM, Pasquale Porreca wrote:

I will briefly explain our use case. This idea is related to another
project to enable the network boot in OpenStack
https://blueprints.launchpad.net/nova/+spec/pxe-boot-instance

We want to make use of the extra-dhcp-opt to indicate as tftp server
a specific instance inside our deployed system, so it will provide
the right operating system to the other instances booting from
network (once the feature from the linked blueprint will be
implemented).

On the tftp server we want to be able to filter what boot file to
provide to different class of instances and our idea was to identify
each class with 2 hexadecimal of the UUID (while the rest would be
random generated, still granting its uniqueness).


It seems like this would still be achievable using the instance tags
feature that Matt mentioned.  And it would be more clear since you
could use human readable class names rather than relying on knowing
that part of the UUID had special meaning.

If you have a need to add specific information to an instance like
'boot class' or want to indicate that an instance in two different
clouds is actually the same one, the Pumphouse use case, that
information should be something we layer on top of an instance and not
something we encode in the UUID.



Anyway this is a customization for our specific environment and for a
feature that is still in early proposal stage, so we wanted to
propose as a separate feature to allow user custom UUID and manage
the generation out of OpenStack.
On 09/24/14 23:15, Matt Riedemann wrote:



On 9/24/2014 3:17 PM, Dean Troyer wrote:

On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka
rpodoly...@mirantis.com mailto:rpodoly...@mirantis.com wrote:

Are there any known gotchas with support of this feature in
REST APIs
(in general)?


I'd be worried about relying on a user-defined attribute in that use
case, that's ripe for a DOS.  Since these are cloud-unique I wouldn't
even need to be in your project to block you from creating that clone
instance if I knew your UUID.

dt

--

Dean Troyer
dtro...@gmail.com mailto:dtro...@gmail.com


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



We talked about this a bit before approving the
'enforce-unique-instance-uuid-in-db' blueprint [1].  As far as we
knew there was no one using null instance UUIDs or duplicates for
that matter.

The instance object already enforces that the UUID field is unique
but the database schema doesn't.  I'll be re-proposing that for Kilo
when it opens up.

If it's a matter of tagging an instance, there is also the tags
blueprint [2] which will probably be proposed again for Kilo.

[1]
https://blueprints.launchpad.net/nova/+spec/enforce-unique-instance-uuid-in-db

[2] https://blueprints.launchpad.net/nova/+spec/tag-instances






___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




If it's a matter of getting the instance tag information down to the 
libvirt driver on boot that shouldn't be a problem, there are others 
asking for similar things, i.e. I want to tag my instances at create 
time and store that tag metadata in some namespace in the libvirt domain 
xml so I can have an application outside of openstack consuming those 
domain xml's and reading that custom namespace information.


--

Thanks,

Matt Riedemann


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-25 Thread Daniel P. Berrange
On Thu, Sep 25, 2014 at 09:19:03AM -0500, Matt Riedemann wrote:
 
 
 On 9/25/2014 8:26 AM, Pasquale Porreca wrote:
 The problem to use a different tag than UUID is that it won't be
 possible (for what I know) to include this tag in the Bootstrap Protocol
 messages exchanged during the pre-boot phase.
 
 Our original idea was to use the Client-identifier (option 61) or Vendor
 class identifier (option 60) of the dhcp request to achieve our target,
 but these fields cannot be controlled in libvirt template and so they
 cannot be set in OpenStack either. Instead the UUID is set it the
 libvirt template by OpenStack and it is included in the messages
 exchanged in the pre-boot phase (option 97) by the instance trying to
 boot from network.
 
 Reference:
 http://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml

[snip]

 If it's a matter of getting the instance tag information down to the libvirt
 driver on boot that shouldn't be a problem, there are others asking for
 similar things, i.e. I want to tag my instances at create time and store
 that tag metadata in some namespace in the libvirt domain xml so I can have
 an application outside of openstack consuming those domain xml's and reading
 that custom namespace information.

Perhaps I'm misunderstanding something, but isn't the DHCP client that
needs to send the tag running in the guest OS ? Libvirt is involved wrt
UUID, because UUID is populated in the guest's virtual BIOS and then
extracted by the guest OS and from there used by the DHCP client. If
we're talking about making a different tag/identifier available for
the DHCP client, then this is probably not going to involve libvirt
unless it also gets pushed up via the virtual BIOS. IOW, couldn't you
just pass whatever tag is needed to the guest OS via the configdrive
or metadata service.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-25 Thread Pasquale Porreca
This is correct Daniel, except that that it is done by the virtual 
firmware/BIOS of the virtual machine and not by the OS (not yet 
installed at that time).


This is the reason we thought about UUID: it is yet used by the iPXE 
client to be included in Bootstrap Protocol messages, it is taken from 
the uuid field in libvirt template and the uuid in libvirt is set by 
OpenStack; the only missing passage is the chance to set the UUID in 
OpenStack instead to have it randomly generated.


Having another user defined tag in libvirt won't help for our issue, 
since it won't be included in Bootstrap Protocol messages, not without 
changes in the virtual BIOS/firmware (as you stated too) and honestly my 
team doesn't have interest in this (neither the competence).


I don't think the configdrive or metadata service would help either: the 
OS on the instance is not yet installed at that time (the target if the 
network boot is exactly to install the OS on the instance!), so it won't 
be able to mount it.


On 09/25/14 16:24, Daniel P. Berrange wrote:

On Thu, Sep 25, 2014 at 09:19:03AM -0500, Matt Riedemann wrote:


On 9/25/2014 8:26 AM, Pasquale Porreca wrote:

The problem to use a different tag than UUID is that it won't be
possible (for what I know) to include this tag in the Bootstrap Protocol
messages exchanged during the pre-boot phase.

Our original idea was to use the Client-identifier (option 61) or Vendor
class identifier (option 60) of the dhcp request to achieve our target,
but these fields cannot be controlled in libvirt template and so they
cannot be set in OpenStack either. Instead the UUID is set it the
libvirt template by OpenStack and it is included in the messages
exchanged in the pre-boot phase (option 97) by the instance trying to
boot from network.

Reference:
http://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml

[snip]


If it's a matter of getting the instance tag information down to the libvirt
driver on boot that shouldn't be a problem, there are others asking for
similar things, i.e. I want to tag my instances at create time and store
that tag metadata in some namespace in the libvirt domain xml so I can have
an application outside of openstack consuming those domain xml's and reading
that custom namespace information.

Perhaps I'm misunderstanding something, but isn't the DHCP client that
needs to send the tag running in the guest OS ? Libvirt is involved wrt
UUID, because UUID is populated in the guest's virtual BIOS and then
extracted by the guest OS and from there used by the DHCP client. If
we're talking about making a different tag/identifier available for
the DHCP client, then this is probably not going to involve libvirt
unless it also gets pushed up via the virtual BIOS. IOW, couldn't you
just pass whatever tag is needed to the guest OS via the configdrive
or metadata service.

Regards,
Daniel


--
Pasquale Porreca

DEK Technologies
Via dei Castelli Romani, 22
00040 Pomezia (Roma)

Mobile +39 3394823805
Skype paskporr


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-25 Thread Daniel P. Berrange
On Thu, Sep 25, 2014 at 05:23:22PM +0200, Pasquale Porreca wrote:
 This is correct Daniel, except that that it is done by the virtual
 firmware/BIOS of the virtual machine and not by the OS (not yet installed at
 that time).
 
 This is the reason we thought about UUID: it is yet used by the iPXE client
 to be included in Bootstrap Protocol messages, it is taken from the uuid
 field in libvirt template and the uuid in libvirt is set by OpenStack; the
 only missing passage is the chance to set the UUID in OpenStack instead to
 have it randomly generated.
 
 Having another user defined tag in libvirt won't help for our issue, since
 it won't be included in Bootstrap Protocol messages, not without changes in
 the virtual BIOS/firmware (as you stated too) and honestly my team doesn't
 have interest in this (neither the competence).
 
 I don't think the configdrive or metadata service would help either: the OS
 on the instance is not yet installed at that time (the target if the network
 boot is exactly to install the OS on the instance!), so it won't be able to
 mount it.

Ok, yes, if we're considering the DHCP client inside the iPXE BIOS
blob, then I don't see any currently viable options besides UUID.
There's no mechanism for passing any other data into iPXE that I
am aware of, though if there is a desire todo that it could be
raised on the QEMU mailing list for discussion.


Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-24 Thread joehuang
+1.

Or at least provide a way to specify an external UUID for the instance, and can 
retrieve the instance through the external UUID which may be linked to external 
system's object.

Chaoyi Huang ( joehuang )

发件人: Pasquale Porreca [pasquale.porr...@dektech.com.au]
发送时间: 2014年9月24日 21:08
收件人: openstack-dev@lists.openstack.org
主题: [openstack-dev]  [nova] Create an instance with a custom uuid

Hello

I would like to be able to specify the UUID of an instance when I create
it. I found this discussion about this matter:
https://lists.launchpad.net/openstack/msg22387.html
but I could not find any blueprint, anyway I understood this
modification should not comport any particular issue.

Would it be acceptable to pass the uuid as metadata, or should I instead
modify the api if I want to set the UUID from the novaclient?

Best regards

--
Pasquale Porreca

DEK Technologies
Via dei Castelli Romani, 22
00040 Pomezia (Roma)

Mobile +39 3394823805
Skype paskporr


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-24 Thread Roman Podoliaka
Hi Joe,

Tools like Pumphouse [1] (migrates workloads, e.g. instances, between
two OpenStack clouds) would benefit from supporting this (Pumphouse
would be able to replicate user instances in a new cloud up to their
UUIDs).

Are there any known gotchas with support of this feature in REST APIs
(in general)?

Thanks,
Roman

[1] https://github.com/MirantisLabs/pumphouse

On Wed, Sep 24, 2014 at 10:23 AM, Joe Gordon joe.gord...@gmail.com wrote:
 Whats the use case for this? We should be thorough when making API changes
 like this.

 On Wed, Sep 24, 2014 at 6:56 AM, joehuang joehu...@huawei.com wrote:

 +1.

 Or at least provide a way to specify an external UUID for the instance,
 and can retrieve the instance through the external UUID which may be linked
 to external system's object.

 Chaoyi Huang ( joehuang )
 
 发件人: Pasquale Porreca [pasquale.porr...@dektech.com.au]
 发送时间: 2014年9月24日 21:08
 收件人: openstack-dev@lists.openstack.org
 主题: [openstack-dev]  [nova] Create an instance with a custom uuid

 Hello

 I would like to be able to specify the UUID of an instance when I create
 it. I found this discussion about this matter:
 https://lists.launchpad.net/openstack/msg22387.html
 but I could not find any blueprint, anyway I understood this
 modification should not comport any particular issue.

 Would it be acceptable to pass the uuid as metadata, or should I instead
 modify the api if I want to set the UUID from the novaclient?

 Best regards

 --
 Pasquale Porreca

 DEK Technologies
 Via dei Castelli Romani, 22
 00040 Pomezia (Roma)

 Mobile +39 3394823805
 Skype paskporr


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-24 Thread Dean Troyer
On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka rpodoly...@mirantis.com
wrote:

 Are there any known gotchas with support of this feature in REST APIs
 (in general)?


I'd be worried about relying on a user-defined attribute in that use case,
that's ripe for a DOS.  Since these are cloud-unique I wouldn't even need
to be in your project to block you from creating that clone instance if I
knew your UUID.

dt

-- 

Dean Troyer
dtro...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-24 Thread Daniel P. Berrange
On Wed, Sep 24, 2014 at 12:58:06PM -0700, Roman Podoliaka wrote:
 Hi Joe,
 
 Tools like Pumphouse [1] (migrates workloads, e.g. instances, between
 two OpenStack clouds) would benefit from supporting this (Pumphouse
 would be able to replicate user instances in a new cloud up to their
 UUIDs).
 
 Are there any known gotchas with support of this feature in REST APIs
 (in general)?

UUIDs must obviously be unique across *all* users' VMs in the cloud, thus
this is obviously a risk that the UUID you request is already in use by
another user account's VM instance. If people use a sensible random UUID
generator this risk is small, but if people decide to use easy to remember
UUIDs like 0----1 then the collision risk is
large :-)

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-24 Thread Matt Riedemann



On 9/24/2014 3:17 PM, Dean Troyer wrote:

On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka
rpodoly...@mirantis.com mailto:rpodoly...@mirantis.com wrote:

Are there any known gotchas with support of this feature in REST APIs
(in general)?


I'd be worried about relying on a user-defined attribute in that use
case, that's ripe for a DOS.  Since these are cloud-unique I wouldn't
even need to be in your project to block you from creating that clone
instance if I knew your UUID.

dt

--

Dean Troyer
dtro...@gmail.com mailto:dtro...@gmail.com


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



We talked about this a bit before approving the 
'enforce-unique-instance-uuid-in-db' blueprint [1].  As far as we knew 
there was no one using null instance UUIDs or duplicates for that matter.


The instance object already enforces that the UUID field is unique but 
the database schema doesn't.  I'll be re-proposing that for Kilo when it 
opens up.


If it's a matter of tagging an instance, there is also the tags 
blueprint [2] which will probably be proposed again for Kilo.


[1] 
https://blueprints.launchpad.net/nova/+spec/enforce-unique-instance-uuid-in-db

[2] https://blueprints.launchpad.net/nova/+spec/tag-instances

--

Thanks,

Matt Riedemann


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Create an instance with a custom uuid

2014-09-24 Thread joehuang
It can also be done by external system to do the mapping of UUID, but not to 
change the API.

When I develop Windows app, for example CListBox, for each list item, you can 
set item-data(four bytes). Sometimes I use the item-data to store the index or 
pointer of the object the item belongs to.

If the user click the list box, and select one item, the regarding object could 
be find very fast.

This is also not good idea for we have to change the API and  “that's ripe for 
a DOS“ (rpodoly...@mirantis.commailto:rpodoly...@mirantis.com).

So I admit that I give +1 too early.  Thanks to Joe and rpodolyaka.

Best Regards
Chaoyi Huang ( Joe Huang )


发件人: Joe Gordon [mailto:joe.gord...@gmail.com]
发送时间: 2014年9月25日 1:23
收件人: OpenStack Development Mailing List (not for usage questions)
主题: Re: [openstack-dev] [nova] Create an instance with a custom uuid

Whats the use case for this? We should be thorough when making API changes like 
this.

On Wed, Sep 24, 2014 at 6:56 AM, joehuang 
joehu...@huawei.commailto:joehu...@huawei.com wrote:
+1.

Or at least provide a way to specify an external UUID for the instance, and can 
retrieve the instance through the external UUID which may be linked to external 
system's object.

Chaoyi Huang ( joehuang )

发件人: Pasquale Porreca 
[pasquale.porr...@dektech.com.aumailto:pasquale.porr...@dektech.com.au]
发送时间: 2014年9月24日 21:08
收件人: openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
主题: [openstack-dev]  [nova] Create an instance with a custom uuid

Hello

I would like to be able to specify the UUID of an instance when I create
it. I found this discussion about this matter:
https://lists.launchpad.net/openstack/msg22387.html
but I could not find any blueprint, anyway I understood this
modification should not comport any particular issue.

Would it be acceptable to pass the uuid as metadata, or should I instead
modify the api if I want to set the UUID from the novaclient?

Best regards

--
Pasquale Porreca

DEK Technologies
Via dei Castelli Romani, 22
00040 Pomezia (Roma)

Mobile +39 3394823805tel:%2B39%203394823805
Skype paskporr


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev