Re: [openstack-dev] [nova] ability to set metadata on instances (but config drive is not updated)
Worse, if you use ironic, I think configdrive is mapped to a partition at provision time. so hot plugging can't ever work in that scenario. Thanks, Kevin From: Clint Byrum [cl...@fewbar.com] Sent: Tuesday, June 21, 2016 4:15 PM To: openstack-dev Subject: Re: [openstack-dev] [nova] ability to set metadata on instances (but config drive is not updated) Excerpts from Joshua Harlow's message of 2016-06-21 15:12:09 -0700: > Agreed, it appears supported right-now (whether intentional or not), > > So the question at that point is what can we do to make it better... > > I think we all agree that the config-drive probably shouldn't have the > equivalent of the metadata service in it; because if the metadata > service can change over time, and the config-drive can't then it's a bad > equivalent and that should be rectified, either by making it an > equivalent or making it be accepted that it is not, and ideally trimming > the data in it to not confuse people (ie by providing static networking > configuration only, and leaving the items that can be dynamic to the > dynamic metadata service). > > Of course this whole mess IMHO gets into, 'why don't we just have an > agreed up agent that can be used'; because that's what the metadata > service is starting to become (a crappy version of an agent); but I digress. > Not sure I agree with your initial stipulation. It shouldn't be a surprise, at all, to any user, that an HTTP service is updated live, while an ISO attached to an instance is kept static. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] ability to set metadata on instances (but config drive is not updated)
> I'm going to have to agree with Jay - and I believe with something Clint > said earlier. > > Use metadata/config-drive to bootstrap you. Then use your actual tools > (puppet/chef/ansible/salt/juju/cfengine) to interact with the node once > it's enrolled in whatever you use to orchestrate cloud things. The best > part about doing that is that your orchestration layer can talk to the > OpenStack APIs AND to your nodes. Most of them also support key-value > pairs, as well as updating them. Yep, agree as well. I hate that metadata is changeable, but I'm not sure there is much we can do about it now because I think at least one "feature" is implemented this way. However, definitely +1 to not adding any more such pain. We've seen plenty of issues around scaling (read: puppet DoSing) the metadata stuff. Caching it forever as a static file and/or using configdrive makes those operations so much more efficient, especially in the face of dumb things like facter constantly banging the crap out of it. Many people ask us to reimplement things like auth, nagios, and config management in nova. That doesn't mean it's something we should do. --Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] ability to set metadata on instances (but config drive is not updated)
Excerpts from Joshua Harlow's message of 2016-06-21 15:12:09 -0700: > Agreed, it appears supported right-now (whether intentional or not), > > So the question at that point is what can we do to make it better... > > I think we all agree that the config-drive probably shouldn't have the > equivalent of the metadata service in it; because if the metadata > service can change over time, and the config-drive can't then it's a bad > equivalent and that should be rectified, either by making it an > equivalent or making it be accepted that it is not, and ideally trimming > the data in it to not confuse people (ie by providing static networking > configuration only, and leaving the items that can be dynamic to the > dynamic metadata service). > > Of course this whole mess IMHO gets into, 'why don't we just have an > agreed up agent that can be used'; because that's what the metadata > service is starting to become (a crappy version of an agent); but I digress. > Not sure I agree with your initial stipulation. It shouldn't be a surprise, at all, to any user, that an HTTP service is updated live, while an ISO attached to an instance is kept static. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] ability to set metadata on instances (but config drive is not updated)
Agreed, it appears supported right-now (whether intentional or not), So the question at that point is what can we do to make it better... I think we all agree that the config-drive probably shouldn't have the equivalent of the metadata service in it; because if the metadata service can change over time, and the config-drive can't then it's a bad equivalent and that should be rectified, either by making it an equivalent or making it be accepted that it is not, and ideally trimming the data in it to not confuse people (ie by providing static networking configuration only, and leaving the items that can be dynamic to the dynamic metadata service). Of course this whole mess IMHO gets into, 'why don't we just have an agreed up agent that can be used'; because that's what the metadata service is starting to become (a crappy version of an agent); but I digress. Chris Friesen wrote: On 06/19/2016 11:57 AM, Clint Byrum wrote: Excerpts from Joshua Harlow's message of 2016-06-17 15:04:38 -0700: Now if I am using the configdrive instead of the metadata server at that special/magic ip that same metadata never seems to change (I assume the configdrive would have to be 'ejected' and then a new configdrive created and then that configdrive 'reinserted'); was anyone aware of a bug that would solve this (it does appear to be a feature difference that could/should? be solved)? That's not a bug. AFAIK, it was never intended to be updated. Why this is something useful (from my view) is that we (at godaddy) have a cron job that polls that metadata periodically and it generates a bunch of polling traffic (especially when each VM does this) and that traffic could be removed if such a 'eject' and 'reinsert' happens instead (since then the cron job could become a small program that listens for devices being inserted/removed and does the needed actions then, which is better than polling endlessly for data that hasn't changed). I think you're abusing Nova to do what your config management system should be doing. The config drive and metadata service are not meant to be continuously polled or updated, they're for bootstrapping into a continuous config management system. While this is true, it seems to be a commonly-requested item. I just had a customer rep asking whether we support updating instance metadata for a running instance, since apparently it's possible with AWS. Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] ability to set metadata on instances (but config drive is not updated)
On 06/19/2016 11:57 AM, Clint Byrum wrote: Excerpts from Joshua Harlow's message of 2016-06-17 15:04:38 -0700: Now if I am using the configdrive instead of the metadata server at that special/magic ip that same metadata never seems to change (I assume the configdrive would have to be 'ejected' and then a new configdrive created and then that configdrive 'reinserted'); was anyone aware of a bug that would solve this (it does appear to be a feature difference that could/should? be solved)? That's not a bug. AFAIK, it was never intended to be updated. Why this is something useful (from my view) is that we (at godaddy) have a cron job that polls that metadata periodically and it generates a bunch of polling traffic (especially when each VM does this) and that traffic could be removed if such a 'eject' and 'reinsert' happens instead (since then the cron job could become a small program that listens for devices being inserted/removed and does the needed actions then, which is better than polling endlessly for data that hasn't changed). I think you're abusing Nova to do what your config management system should be doing. The config drive and metadata service are not meant to be continuously polled or updated, they're for bootstrapping into a continuous config management system. While this is true, it seems to be a commonly-requested item. I just had a customer rep asking whether we support updating instance metadata for a running instance, since apparently it's possible with AWS. Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] ability to set metadata on instances (but config drive is not updated)
On 06/21/2016 10:06 AM, Jay Faulkner wrote:
>
>
> On 6/17/16 3:04 PM, Joshua Harlow wrote:
>> Hi folks,
>>
>> I was noticing that its possible to do something like:
>>
>> $ nova meta josh-testr3 set "e=f"
>>
>> Then inside the VM I can do the following to eventually see that this
>> changes shows up in the instance metadata exposed at the following:
>>
>> $ curl -s http://169.254.169.254/openstack/latest/meta_data.json |
>> python -mjson.tool
>>
>> {
>> ...
>> "hostname": "josh-testr3.cloud.phx3.gdg",
>> "launch_index": 0,
>> "meta": {
>> ...
>> "e": "f",
>> ..
>> }
>> ...
>> }
>>
>> Now if I am using the configdrive instead of the metadata server at
>> that special/magic ip that same metadata never seems to change (I
>> assume the configdrive would have to be 'ejected' and then a new
>> configdrive created and then that configdrive 'reinserted'); was
>> anyone aware of a bug that would solve this (it does appear to be a
>> feature difference that could/should? be solved)?
>>
> I would be -1 to instituting this change as well, as it would be
> impossible for some hypervisors/drivers (such as Ironic) to implement.
> Additionally, how could you ensure the tenant OS didn't have the
> configdrive mounted or otherwise in use?
I'm going to have to agree with Jay - and I believe with something Clint
said earlier.
Use metadata/config-drive to bootstrap you. Then use your actual tools
(puppet/chef/ansible/salt/juju/cfengine) to interact with the node once
it's enrolled in whatever you use to orchestrate cloud things. The best
part about doing that is that your orchestration layer can talk to the
OpenStack APIs AND to your nodes. Most of them also support key-value
pairs, as well as updating them.
>
>> Why this is something useful (from my view) is that we (at godaddy)
>> have a cron job that polls that metadata periodically and it generates
>> a bunch of polling traffic (especially when each VM does this) and
>> that traffic could be removed if such a 'eject' and 'reinsert' happens
>> instead (since then the cron job could become a small program that
>> listens for devices being inserted/removed and does the needed actions
>> then, which is better than polling endlessly for data that hasn't
>> changed).
>>
>> -Josh
>>
>> __
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] ability to set metadata on instances (but config drive is not updated)
On 6/17/16 3:04 PM, Joshua Harlow wrote:
Hi folks,
I was noticing that its possible to do something like:
$ nova meta josh-testr3 set "e=f"
Then inside the VM I can do the following to eventually see that this
changes shows up in the instance metadata exposed at the following:
$ curl -s http://169.254.169.254/openstack/latest/meta_data.json |
python -mjson.tool
{
...
"hostname": "josh-testr3.cloud.phx3.gdg",
"launch_index": 0,
"meta": {
...
"e": "f",
..
}
...
}
Now if I am using the configdrive instead of the metadata server at
that special/magic ip that same metadata never seems to change (I
assume the configdrive would have to be 'ejected' and then a new
configdrive created and then that configdrive 'reinserted'); was
anyone aware of a bug that would solve this (it does appear to be a
feature difference that could/should? be solved)?
I would be -1 to instituting this change as well, as it would be
impossible for some hypervisors/drivers (such as Ironic) to implement.
Additionally, how could you ensure the tenant OS didn't have the
configdrive mounted or otherwise in use?
Thanks,
Jay
Why this is something useful (from my view) is that we (at godaddy)
have a cron job that polls that metadata periodically and it generates
a bunch of polling traffic (especially when each VM does this) and
that traffic could be removed if such a 'eject' and 'reinsert' happens
instead (since then the cron job could become a small program that
listens for devices being inserted/removed and does the needed actions
then, which is better than polling endlessly for data that hasn't
changed).
-Josh
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] ability to set metadata on instances (but config drive is not updated)
Excerpts from Joshua Harlow's message of 2016-06-17 15:04:38 -0700:
> Hi folks,
>
> I was noticing that its possible to do something like:
>
> $ nova meta josh-testr3 set "e=f"
>
> Then inside the VM I can do the following to eventually see that this
> changes shows up in the instance metadata exposed at the following:
>
> $ curl -s http://169.254.169.254/openstack/latest/meta_data.json |
> python -mjson.tool
>
> {
> ...
> "hostname": "josh-testr3.cloud.phx3.gdg",
> "launch_index": 0,
> "meta": {
> ...
> "e": "f",
> ..
> }
> ...
> }
>
> Now if I am using the configdrive instead of the metadata server at that
> special/magic ip that same metadata never seems to change (I assume the
> configdrive would have to be 'ejected' and then a new configdrive
> created and then that configdrive 'reinserted'); was anyone aware of a
> bug that would solve this (it does appear to be a feature difference
> that could/should? be solved)?
>
That's not a bug. AFAIK, it was never intended to be updated.
> Why this is something useful (from my view) is that we (at godaddy) have
> a cron job that polls that metadata periodically and it generates a
> bunch of polling traffic (especially when each VM does this) and that
> traffic could be removed if such a 'eject' and 'reinsert' happens
> instead (since then the cron job could become a small program that
> listens for devices being inserted/removed and does the needed actions
> then, which is better than polling endlessly for data that hasn't changed).
I think you're abusing Nova to do what your config management system
should be doing. The config drive and metadata service are not meant
to be continuously polled or updated, they're for bootstrapping into
a continuous config management system.
I'd suggest putting your metadata into something that is appropriately
scaled and doesn't have to do backflips to figure out your instance
ID by looking up your network address in another service, or using a
push technology like Ansible or mcollective (does anybody still use
that?) which doesn't suffer from needless polling.
Could we make Nova do this? Yes. Do we need to? No, It's solved by
everyone else just fine without hypervisor collaboration, and Nova has
more important things to do.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] ability to set metadata on instances (but config drive is not updated)
Hi folks,
I was noticing that its possible to do something like:
$ nova meta josh-testr3 set "e=f"
Then inside the VM I can do the following to eventually see that this
changes shows up in the instance metadata exposed at the following:
$ curl -s http://169.254.169.254/openstack/latest/meta_data.json |
python -mjson.tool
{
...
"hostname": "josh-testr3.cloud.phx3.gdg",
"launch_index": 0,
"meta": {
...
"e": "f",
..
}
...
}
Now if I am using the configdrive instead of the metadata server at that
special/magic ip that same metadata never seems to change (I assume the
configdrive would have to be 'ejected' and then a new configdrive
created and then that configdrive 'reinserted'); was anyone aware of a
bug that would solve this (it does appear to be a feature difference
that could/should? be solved)?
Why this is something useful (from my view) is that we (at godaddy) have
a cron job that polls that metadata periodically and it generates a
bunch of polling traffic (especially when each VM does this) and that
traffic could be removed if such a 'eject' and 'reinsert' happens
instead (since then the cron job could become a small program that
listens for devices being inserted/removed and does the needed actions
then, which is better than polling endlessly for data that hasn't changed).
-Josh
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
