Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Dan Smith]

>> The current sticking point is that nova-core doesn't want to pass
>> system-metadata to plugins, as the internal format of that changes
>> between releases and isn't intended to be versioned. Instead, we want to
>> pass just the specific bits people need.
>>
>> So -- what things are you using from system-metadata specifically?
> 
> Right now just image_os_distro and image_os_version.

Which may very well go away in the near future because of other ways we
handle this.

IMHO, including system_metadata should not be on the table. If you need
things about the image, you should go ask the image service for that
information, given what we tell you about the instance (i.e. image-id in
this case). If you want to chase our internals, then dig it out of the
database. Looking at system_metadata is looking at our internals, just
like poking in our database. It's not something we should (effectively)
codify in this interface because we WILL break it.

--Dan

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Michael Still]

Just an update on this work.

The punch line is that I think this code is unlikely to land in Newton. It
has until Wednesday to be finalized, and I don't see that happening in
time. That said, I'm still _trying_ to land it.

The current sticking point is that nova-core doesn't want to pass
system-metadata to plugins, as the internal format of that changes between
releases and isn't intended to be versioned. Instead, we want to pass just
the specific bits people need.

So -- what things are you using from system-metadata specifically?

Michael




On Fri, May 13, 2016 at 10:15 PM, Joseph Bajin <josephba...@gmail.com>
wrote:

> There is also a golang library that can validate tokens..
>
> http://gophercloud.io/docs/identity/v3/
>
>
>
> On Thu, May 12, 2016 at 11:25 PM, David Medberry <openst...@medberry.net>
> wrote:
>
>> There's a jython implementation of keystone and I thought there was other
>> work to validate tokens from within Java. Added Jim Baker to the thread.
>>
>> -d
>>
>> On Thu, May 12, 2016 at 5:06 PM, Michael Still <mi...@stillhq.com> wrote:
>>
>>> I'm just going to reply to myself here with another status update.
>>>
>>> The design seems largely settled at this point, with one exception --
>>> how does nova authenticate with the external microservice?
>>>
>>> The current proposal is to have nova use the client's keystone token to
>>> authenticate with the external microservice. This is a neat solution
>>> because its what nova does when talking to other services in your OpenStack
>>> deployment, so its consistent and well understood.
>>>
>>> The catch here is that it means your external microservice needs to know
>>> how to do keystone authentication. That's well understood for python
>>> microservices, and I can provide sample code for that case using the
>>> keystone wsgi middleware. On the other hand, its harder for things like
>>> Java where I'm not sure I'm aware of any keystone auth implementation. Is
>>> effectively requiring the microservices to be written in python a
>>> particular problem? I'm hoping not given that all the current plugins are
>>> written in python by definition.
>>>
>>> Cheers,
>>> Michael
>>>
>>>
>>>
>>>
>>> On Wed, May 4, 2016 at 7:37 AM, Michael Still <mi...@stillhq.com> wrote:
>>>
>>>> Hey,
>>>>
>>>> I just wanted to let people know that the review is progressing, but we
>>>> have a question.
>>>>
>>>> Do operators really need to call more than one external REST service to
>>>> collect vendordata? We can implement that in nova, but it would be nice to
>>>> reduce the complexity to only having one external REST service. If you
>>>> needed to call more than one service you could of course write a REST
>>>> service that aggregated REST services.
>>>>
>>>> Does anyone in the operator community have strong feelings either way?
>>>> Should nova be able to call more than one external vendordata REST service?
>>>>
>>>> Thanks,
>>>> Michael
>>>>
>>>>
>>>>
>>>>
>>>> On Sat, Apr 30, 2016 at 4:11 AM, Michael Still <mi...@stillhq.com>
>>>> wrote:
>>>>
>>>>> So, after a series of hallway track chats this week, I wrote this:
>>>>>
>>>>> https://review.openstack.org/#/c/310904/
>>>>>
>>>>> Which is a proposal for how to implement vendordata in a way which
>>>>> would (probably) be acceptable to nova, whilst also meeting the needs of
>>>>> operators. I should reinforce that because this week is so hectic nova 
>>>>> core
>>>>> hasn't really talked about this yet, but I am pretty sure I understand and
>>>>> have addressed Sean's concerns.
>>>>>
>>>>> I'd be curious as to if the proposed solution actually meets your
>>>>> needs.
>>>>>
>>>>> Michael
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Apr 18, 2016 at 10:55 AM, Fox, Kevin M <kevin@pnnl.gov>
>>>>> wrote:
>>>>>
>>>>>> We've used it too to work around the lack of instance users in nova.
>>>>>> Please keep it until a viable solution can be reached.
>>>>>>
>>>>>> Thanks,
>>>>>> Kevin
>>>

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Joseph Bajin]

There is also a golang library that can validate tokens..

http://gophercloud.io/docs/identity/v3/



On Thu, May 12, 2016 at 11:25 PM, David Medberry <openst...@medberry.net>
wrote:

> There's a jython implementation of keystone and I thought there was other
> work to validate tokens from within Java. Added Jim Baker to the thread.
>
> -d
>
> On Thu, May 12, 2016 at 5:06 PM, Michael Still <mi...@stillhq.com> wrote:
>
>> I'm just going to reply to myself here with another status update.
>>
>> The design seems largely settled at this point, with one exception -- how
>> does nova authenticate with the external microservice?
>>
>> The current proposal is to have nova use the client's keystone token to
>> authenticate with the external microservice. This is a neat solution
>> because its what nova does when talking to other services in your OpenStack
>> deployment, so its consistent and well understood.
>>
>> The catch here is that it means your external microservice needs to know
>> how to do keystone authentication. That's well understood for python
>> microservices, and I can provide sample code for that case using the
>> keystone wsgi middleware. On the other hand, its harder for things like
>> Java where I'm not sure I'm aware of any keystone auth implementation. Is
>> effectively requiring the microservices to be written in python a
>> particular problem? I'm hoping not given that all the current plugins are
>> written in python by definition.
>>
>> Cheers,
>> Michael
>>
>>
>>
>>
>> On Wed, May 4, 2016 at 7:37 AM, Michael Still <mi...@stillhq.com> wrote:
>>
>>> Hey,
>>>
>>> I just wanted to let people know that the review is progressing, but we
>>> have a question.
>>>
>>> Do operators really need to call more than one external REST service to
>>> collect vendordata? We can implement that in nova, but it would be nice to
>>> reduce the complexity to only having one external REST service. If you
>>> needed to call more than one service you could of course write a REST
>>> service that aggregated REST services.
>>>
>>> Does anyone in the operator community have strong feelings either way?
>>> Should nova be able to call more than one external vendordata REST service?
>>>
>>> Thanks,
>>> Michael
>>>
>>>
>>>
>>>
>>> On Sat, Apr 30, 2016 at 4:11 AM, Michael Still <mi...@stillhq.com>
>>> wrote:
>>>
>>>> So, after a series of hallway track chats this week, I wrote this:
>>>>
>>>> https://review.openstack.org/#/c/310904/
>>>>
>>>> Which is a proposal for how to implement vendordata in a way which
>>>> would (probably) be acceptable to nova, whilst also meeting the needs of
>>>> operators. I should reinforce that because this week is so hectic nova core
>>>> hasn't really talked about this yet, but I am pretty sure I understand and
>>>> have addressed Sean's concerns.
>>>>
>>>> I'd be curious as to if the proposed solution actually meets your needs.
>>>>
>>>> Michael
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Apr 18, 2016 at 10:55 AM, Fox, Kevin M <kevin@pnnl.gov>
>>>> wrote:
>>>>
>>>>> We've used it too to work around the lack of instance users in nova.
>>>>> Please keep it until a viable solution can be reached.
>>>>>
>>>>> Thanks,
>>>>> Kevin
>>>>> --
>>>>> *From:* David Medberry [openst...@medberry.net]
>>>>> *Sent:* Monday, April 18, 2016 7:16 AM
>>>>> *To:* Ned Rhudy
>>>>> *Cc:* openstack-operators@lists.openstack.org
>>>>>
>>>>> *Subject:* Re: [Openstack-operators] Anyone else use
>>>>> vendordata_driver in nova.conf?
>>>>>
>>>>> Hi Ned, Jay,
>>>>>
>>>>> We use it also and I have to agree, it's onerous to require users to
>>>>> add that functionality back in. Where was this discussed?
>>>>>
>>>>> On Mon, Apr 18, 2016 at 8:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) <
>>>>> erh...@bloomberg.net> wrote:
>>>>>
>>>>>> Requiring users to remember to pass specific userdata through to
>>>>>> their instance at every launch in order to replace functionality that
>>>>>

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Michael Still]

I'm just going to reply to myself here with another status update.

The design seems largely settled at this point, with one exception -- how
does nova authenticate with the external microservice?

The current proposal is to have nova use the client's keystone token to
authenticate with the external microservice. This is a neat solution
because its what nova does when talking to other services in your OpenStack
deployment, so its consistent and well understood.

The catch here is that it means your external microservice needs to know
how to do keystone authentication. That's well understood for python
microservices, and I can provide sample code for that case using the
keystone wsgi middleware. On the other hand, its harder for things like
Java where I'm not sure I'm aware of any keystone auth implementation. Is
effectively requiring the microservices to be written in python a
particular problem? I'm hoping not given that all the current plugins are
written in python by definition.

Cheers,
Michael




On Wed, May 4, 2016 at 7:37 AM, Michael Still <mi...@stillhq.com> wrote:

> Hey,
>
> I just wanted to let people know that the review is progressing, but we
> have a question.
>
> Do operators really need to call more than one external REST service to
> collect vendordata? We can implement that in nova, but it would be nice to
> reduce the complexity to only having one external REST service. If you
> needed to call more than one service you could of course write a REST
> service that aggregated REST services.
>
> Does anyone in the operator community have strong feelings either way?
> Should nova be able to call more than one external vendordata REST service?
>
> Thanks,
> Michael
>
>
>
>
> On Sat, Apr 30, 2016 at 4:11 AM, Michael Still <mi...@stillhq.com> wrote:
>
>> So, after a series of hallway track chats this week, I wrote this:
>>
>> https://review.openstack.org/#/c/310904/
>>
>> Which is a proposal for how to implement vendordata in a way which would
>> (probably) be acceptable to nova, whilst also meeting the needs of
>> operators. I should reinforce that because this week is so hectic nova core
>> hasn't really talked about this yet, but I am pretty sure I understand and
>> have addressed Sean's concerns.
>>
>> I'd be curious as to if the proposed solution actually meets your needs.
>>
>> Michael
>>
>>
>>
>>
>> On Mon, Apr 18, 2016 at 10:55 AM, Fox, Kevin M <kevin@pnnl.gov>
>> wrote:
>>
>>> We've used it too to work around the lack of instance users in nova.
>>> Please keep it until a viable solution can be reached.
>>>
>>> Thanks,
>>> Kevin
>>> --
>>> *From:* David Medberry [openst...@medberry.net]
>>> *Sent:* Monday, April 18, 2016 7:16 AM
>>> *To:* Ned Rhudy
>>> *Cc:* openstack-operators@lists.openstack.org
>>>
>>> *Subject:* Re: [Openstack-operators] Anyone else use vendordata_driver
>>> in nova.conf?
>>>
>>> Hi Ned, Jay,
>>>
>>> We use it also and I have to agree, it's onerous to require users to add
>>> that functionality back in. Where was this discussed?
>>>
>>> On Mon, Apr 18, 2016 at 8:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) <
>>> erh...@bloomberg.net> wrote:
>>>
>>>> Requiring users to remember to pass specific userdata through to their
>>>> instance at every launch in order to replace functionality that currently
>>>> works invisible to them would be a step backwards. It's an alternative,
>>>> yes, but it's an alternative that adds burden to our users and is not one
>>>> we would pursue.
>>>>
>>>> What is the rationale for desiring to remove this functionality?
>>>>
>>>> From: jaypi...@gmail.com
>>>> Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in
>>>> nova.conf?
>>>>
>>>> On 04/18/2016 09:24 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
>>>> > I noticed while reading through Mitaka release notes that
>>>> > vendordata_driver has been deprecated in Mitaka
>>>> > (https://review.openstack.org/#/c/288107/) and is slated for removal
>>>> at
>>>> > some point. This came as somewhat of a surprise to me - I searched
>>>> > openstack-dev for vendordata-related subject lines going back to
>>>> January
>>>> > and found no discussion on the matter (IRC logs, while available on
>>>> > eavesdrop, are not trivially searchable without a little scripting to
>>

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Michael Still]

On Wed, May 4, 2016 at 8:28 AM, Fox, Kevin M  wrote:

> For a tenant though, I may not want to have to write user-data to bind
> every thing I launch through horizon's nova workflow, heat, sahara, etc.
> Just having one place to put the hook and its always called has some major
> advantages.


To keep the scope of the current proposal manageable, I want to ignore
tenant data for now. I think it could probably be implemented in a
microservice regardless.

The deadline to get this spec approved is May 30 for it to make it into
Newton, so we need to keep scope changes to a minimum if possible.

Michael

-- 
Rackspace Australia
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Fox, Kevin M]

For a tenant though, I may not want to have to write user-data to bind every 
thing I launch through horizon's nova workflow, heat, sahara, etc. Just having 
one place to put the hook and its always called has some major advantages.

Thanks,
Kevin

From: Mathieu Gagné [mga...@calavera.ca]
Sent: Tuesday, May 03, 2016 3:25 PM
To: Fox, Kevin M
Cc: Michael Still; openstack-operators@lists.openstack.org; Sean Dague
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?

On Tue, May 3, 2016 at 5:51 PM, Fox, Kevin M <kevin@pnnl.gov> wrote:
>
> I think I see at least one use case for minimum 2 hooks...
>
> Cloud provider wants to inject some stuff.
>
> Cloud tenant wants their own hook called to inject stuff to point to the
> Config Management server in their own tenant.
>
> Maybe that's not vendor_data but tenant_data or something...
>

I think this could very well be addressed by cloud-init or any other
sane initialization service.
We just have to make sure instance identification and any other
reasonable information are made available to those tools so they can
be passed to their own Config Management server.
I see vendor_data as a non-intrusive way to pass additional data to
the instance without requiring the vendor/provider to inject an agent
or custom code within the customer's image.

--
Mathieu


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Mathieu Gagné]

On Tue, May 3, 2016 at 5:51 PM, Fox, Kevin M  wrote:
>
> I think I see at least one use case for minimum 2 hooks...
>
> Cloud provider wants to inject some stuff.
>
> Cloud tenant wants their own hook called to inject stuff to point to the
> Config Management server in their own tenant.
>
> Maybe that's not vendor_data but tenant_data or something...
>

I think this could very well be addressed by cloud-init or any other
sane initialization service.
We just have to make sure instance identification and any other
reasonable information are made available to those tools so they can
be passed to their own Config Management server.
I see vendor_data as a non-intrusive way to pass additional data to
the instance without requiring the vendor/provider to inject an agent
or custom code within the customer's image.

--
Mathieu

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Fox, Kevin M]

Depends on what its used for... I can see it potentially being used with Chef 
or Puppet, for calling hooks into AD to bind to a domain. etc. Probably at the 
same time. We use it with our keyserver (something similar to Barbican but 
created before Barbican was a thing) to relay trust info between Nova and the 
Keyserver through the Instance.

I've done some careful inheriting of our vendor_data plug-in to get all the 
features in one plugin, but I could see it being difficult for some folks when 
more features are added.

I think I see at least one use case for minimum 2 hooks...

Cloud provider wants to inject some stuff.

Cloud tenant wants their own hook called to inject stuff to point to the Config 
Management server in their own tenant.

Maybe that's not vendor_data but tenant_data or something...

Thanks,
Kevin

From: mikalst...@gmail.com [mikalst...@gmail.com] on behalf of Michael Still 
[mi...@stillhq.com]
Sent: Tuesday, May 03, 2016 2:37 PM
To: Fox, Kevin M
Cc: David Medberry; Ned Rhudy; openstack-operators@lists.openstack.org; Sean 
Dague
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?

Hey,

I just wanted to let people know that the review is progressing, but we have a 
question.

Do operators really need to call more than one external REST service to collect 
vendordata? We can implement that in nova, but it would be nice to reduce the 
complexity to only having one external REST service. If you needed to call more 
than one service you could of course write a REST service that aggregated REST 
services.

Does anyone in the operator community have strong feelings either way? Should 
nova be able to call more than one external vendordata REST service?

Thanks,
Michael




On Sat, Apr 30, 2016 at 4:11 AM, Michael Still 
<mi...@stillhq.com<mailto:mi...@stillhq.com>> wrote:
So, after a series of hallway track chats this week, I wrote this:

https://review.openstack.org/#/c/310904/

Which is a proposal for how to implement vendordata in a way which would 
(probably) be acceptable to nova, whilst also meeting the needs of operators. I 
should reinforce that because this week is so hectic nova core hasn't really 
talked about this yet, but I am pretty sure I understand and have addressed 
Sean's concerns.

I'd be curious as to if the proposed solution actually meets your needs.

Michael




On Mon, Apr 18, 2016 at 10:55 AM, Fox, Kevin M 
<kevin@pnnl.gov<mailto:kevin@pnnl.gov>> wrote:
We've used it too to work around the lack of instance users in nova. Please 
keep it until a viable solution can be reached.

Thanks,
Kevin

From: David Medberry [openst...@medberry.net<mailto:openst...@medberry.net>]
Sent: Monday, April 18, 2016 7:16 AM
To: Ned Rhudy
Cc: 
openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>

Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?

Hi Ned, Jay,

We use it also and I have to agree, it's onerous to require users to add that 
functionality back in. Where was this discussed?

On Mon, Apr 18, 2016 at 8:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) 
<erh...@bloomberg.net<mailto:erh...@bloomberg.net>> wrote:
Requiring users to remember to pass specific userdata through to their instance 
at every launch in order to replace functionality that currently works 
invisible to them would be a step backwards. It's an alternative, yes, but it's 
an alternative that adds burden to our users and is not one we would pursue.

What is the rationale for desiring to remove this functionality?

From: jaypi...@gmail.com<mailto:jaypi...@gmail.com>
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?
On 04/18/2016 09:24 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
> I noticed while reading through Mitaka release notes that
> vendordata_driver has been deprecated in Mitaka
> (https://review.openstack.org/#/c/288107/) and is slated for removal at
> some point. This came as somewhat of a surprise to me - I searched
> openstack-dev for vendordata-related subject lines going back to January
> and found no discussion on the matter (IRC logs, while available on
> eavesdrop, are not trivially searchable without a little scripting to
> fetch them first, so I didn't check there yet).
>
> We at Bloomberg make heavy use of this particular feature to inject
> dynamically generated JSON into the metadata service of instances; the
> content of the JSON differs depending on the instance making the request
> to the metadata service. The functionality that adds the contents of a
> static JSON file, while remaining around, is not suitable for our use case.
>
> Please let me know if you use vendordata_driver so that I/we can present
> an organized case for why this option or equivalent functionality needs
> to rem

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Michael Still]

Hey,

I just wanted to let people know that the review is progressing, but we
have a question.

Do operators really need to call more than one external REST service to
collect vendordata? We can implement that in nova, but it would be nice to
reduce the complexity to only having one external REST service. If you
needed to call more than one service you could of course write a REST
service that aggregated REST services.

Does anyone in the operator community have strong feelings either way?
Should nova be able to call more than one external vendordata REST service?

Thanks,
Michael




On Sat, Apr 30, 2016 at 4:11 AM, Michael Still <mi...@stillhq.com> wrote:

> So, after a series of hallway track chats this week, I wrote this:
>
> https://review.openstack.org/#/c/310904/
>
> Which is a proposal for how to implement vendordata in a way which would
> (probably) be acceptable to nova, whilst also meeting the needs of
> operators. I should reinforce that because this week is so hectic nova core
> hasn't really talked about this yet, but I am pretty sure I understand and
> have addressed Sean's concerns.
>
> I'd be curious as to if the proposed solution actually meets your needs.
>
> Michael
>
>
>
>
> On Mon, Apr 18, 2016 at 10:55 AM, Fox, Kevin M <kevin@pnnl.gov> wrote:
>
>> We've used it too to work around the lack of instance users in nova.
>> Please keep it until a viable solution can be reached.
>>
>> Thanks,
>> Kevin
>> --
>> *From:* David Medberry [openst...@medberry.net]
>> *Sent:* Monday, April 18, 2016 7:16 AM
>> *To:* Ned Rhudy
>> *Cc:* openstack-operators@lists.openstack.org
>>
>> *Subject:* Re: [Openstack-operators] Anyone else use vendordata_driver
>> in nova.conf?
>>
>> Hi Ned, Jay,
>>
>> We use it also and I have to agree, it's onerous to require users to add
>> that functionality back in. Where was this discussed?
>>
>> On Mon, Apr 18, 2016 at 8:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) <
>> erh...@bloomberg.net> wrote:
>>
>>> Requiring users to remember to pass specific userdata through to their
>>> instance at every launch in order to replace functionality that currently
>>> works invisible to them would be a step backwards. It's an alternative,
>>> yes, but it's an alternative that adds burden to our users and is not one
>>> we would pursue.
>>>
>>> What is the rationale for desiring to remove this functionality?
>>>
>>> From: jaypi...@gmail.com
>>> Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in
>>> nova.conf?
>>>
>>> On 04/18/2016 09:24 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
>>> > I noticed while reading through Mitaka release notes that
>>> > vendordata_driver has been deprecated in Mitaka
>>> > (https://review.openstack.org/#/c/288107/) and is slated for removal
>>> at
>>> > some point. This came as somewhat of a surprise to me - I searched
>>> > openstack-dev for vendordata-related subject lines going back to
>>> January
>>> > and found no discussion on the matter (IRC logs, while available on
>>> > eavesdrop, are not trivially searchable without a little scripting to
>>> > fetch them first, so I didn't check there yet).
>>> >
>>> > We at Bloomberg make heavy use of this particular feature to inject
>>> > dynamically generated JSON into the metadata service of instances; the
>>> > content of the JSON differs depending on the instance making the
>>> request
>>> > to the metadata service. The functionality that adds the contents of a
>>> > static JSON file, while remaining around, is not suitable for our use
>>> case.
>>> >
>>> > Please let me know if you use vendordata_driver so that I/we can
>>> present
>>> > an organized case for why this option or equivalent functionality needs
>>> > to remain around. The alternative is that we end up patching the
>>> > vendordata driver directly in Nova when we move to Mitaka, which I'd
>>> > like to avoid; as a matter of principle I would rather see more
>>> > classloader overrides, not fewer.
>>>
>>> Wouldn't an alternative be to use something like Chef, Puppet, Ansible,
>>> Saltstack, etc and their associated config variable storage services
>>> like Hiera or something similar to publish custom metadata? That way,
>>> all you need to pass to your instance (via userdata) is a URI or
>>> connection string and some 

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Michael Still]

So, after a series of hallway track chats this week, I wrote this:

https://review.openstack.org/#/c/310904/

Which is a proposal for how to implement vendordata in a way which would
(probably) be acceptable to nova, whilst also meeting the needs of
operators. I should reinforce that because this week is so hectic nova core
hasn't really talked about this yet, but I am pretty sure I understand and
have addressed Sean's concerns.

I'd be curious as to if the proposed solution actually meets your needs.

Michael




On Mon, Apr 18, 2016 at 10:55 AM, Fox, Kevin M <kevin@pnnl.gov> wrote:

> We've used it too to work around the lack of instance users in nova.
> Please keep it until a viable solution can be reached.
>
> Thanks,
> Kevin
> --
> *From:* David Medberry [openst...@medberry.net]
> *Sent:* Monday, April 18, 2016 7:16 AM
> *To:* Ned Rhudy
> *Cc:* openstack-operators@lists.openstack.org
>
> *Subject:* Re: [Openstack-operators] Anyone else use vendordata_driver in
> nova.conf?
>
> Hi Ned, Jay,
>
> We use it also and I have to agree, it's onerous to require users to add
> that functionality back in. Where was this discussed?
>
> On Mon, Apr 18, 2016 at 8:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) <
> erh...@bloomberg.net> wrote:
>
>> Requiring users to remember to pass specific userdata through to their
>> instance at every launch in order to replace functionality that currently
>> works invisible to them would be a step backwards. It's an alternative,
>> yes, but it's an alternative that adds burden to our users and is not one
>> we would pursue.
>>
>> What is the rationale for desiring to remove this functionality?
>>
>> From: jaypi...@gmail.com
>> Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in
>> nova.conf?
>>
>> On 04/18/2016 09:24 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
>> > I noticed while reading through Mitaka release notes that
>> > vendordata_driver has been deprecated in Mitaka
>> > (https://review.openstack.org/#/c/288107/) and is slated for removal at
>> > some point. This came as somewhat of a surprise to me - I searched
>> > openstack-dev for vendordata-related subject lines going back to January
>> > and found no discussion on the matter (IRC logs, while available on
>> > eavesdrop, are not trivially searchable without a little scripting to
>> > fetch them first, so I didn't check there yet).
>> >
>> > We at Bloomberg make heavy use of this particular feature to inject
>> > dynamically generated JSON into the metadata service of instances; the
>> > content of the JSON differs depending on the instance making the request
>> > to the metadata service. The functionality that adds the contents of a
>> > static JSON file, while remaining around, is not suitable for our use
>> case.
>> >
>> > Please let me know if you use vendordata_driver so that I/we can present
>> > an organized case for why this option or equivalent functionality needs
>> > to remain around. The alternative is that we end up patching the
>> > vendordata driver directly in Nova when we move to Mitaka, which I'd
>> > like to avoid; as a matter of principle I would rather see more
>> > classloader overrides, not fewer.
>>
>> Wouldn't an alternative be to use something like Chef, Puppet, Ansible,
>> Saltstack, etc and their associated config variable storage services
>> like Hiera or something similar to publish custom metadata? That way,
>> all you need to pass to your instance (via userdata) is a URI or
>> connection string and some auth details for your config storage service
>> and the instance can grab whatever you need.
>>
>> Thoughts?
>> -jay
>>
>> ___
>> OpenStack-operators mailing list
>> OpenStack-operators@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>>
>>
>> ___
>> OpenStack-operators mailing list
>> OpenStack-operators@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>


-- 
Rackspace Australia
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Fox, Kevin M]

We've used it too to work around the lack of instance users in nova. Please 
keep it until a viable solution can be reached.

Thanks,
Kevin

From: David Medberry [openst...@medberry.net]
Sent: Monday, April 18, 2016 7:16 AM
To: Ned Rhudy
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?

Hi Ned, Jay,

We use it also and I have to agree, it's onerous to require users to add that 
functionality back in. Where was this discussed?

On Mon, Apr 18, 2016 at 8:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) 
<erh...@bloomberg.net<mailto:erh...@bloomberg.net>> wrote:
Requiring users to remember to pass specific userdata through to their instance 
at every launch in order to replace functionality that currently works 
invisible to them would be a step backwards. It's an alternative, yes, but it's 
an alternative that adds burden to our users and is not one we would pursue.

What is the rationale for desiring to remove this functionality?

From: jaypi...@gmail.com<mailto:jaypi...@gmail.com>
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?
On 04/18/2016 09:24 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
> I noticed while reading through Mitaka release notes that
> vendordata_driver has been deprecated in Mitaka
> (https://review.openstack.org/#/c/288107/) and is slated for removal at
> some point. This came as somewhat of a surprise to me - I searched
> openstack-dev for vendordata-related subject lines going back to January
> and found no discussion on the matter (IRC logs, while available on
> eavesdrop, are not trivially searchable without a little scripting to
> fetch them first, so I didn't check there yet).
>
> We at Bloomberg make heavy use of this particular feature to inject
> dynamically generated JSON into the metadata service of instances; the
> content of the JSON differs depending on the instance making the request
> to the metadata service. The functionality that adds the contents of a
> static JSON file, while remaining around, is not suitable for our use case.
>
> Please let me know if you use vendordata_driver so that I/we can present
> an organized case for why this option or equivalent functionality needs
> to remain around. The alternative is that we end up patching the
> vendordata driver directly in Nova when we move to Mitaka, which I'd
> like to avoid; as a matter of principle I would rather see more
> classloader overrides, not fewer.

Wouldn't an alternative be to use something like Chef, Puppet, Ansible,
Saltstack, etc and their associated config variable storage services
like Hiera or something similar to publish custom metadata? That way,
all you need to pass to your instance (via userdata) is a URI or
connection string and some auth details for your config storage service
and the instance can grab whatever you need.

Thoughts?
-jay

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org<mailto:OpenStack-operators@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org<mailto:OpenStack-operators@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Ned Rhudy (BLOOMBERG/ 731 LEX)]

Okay, if I propose something upstream what is it expected to look like? There 
is apparently a high level of opinionation around exposed class loaders I 
wasn't aware of, and I don't think there's any one-size-fits-all solution here. 
If I suggested something like adding additional instance metadata under 
/openstack/latest/meta_data.json, that might be suitable for us as private 
cloud operators but pose security risks to public cloud operators. I don't want 
to propose something that sucks and has Bloomberg pathologies all over it but 
gets jackhammered in anyway.

From: s...@dague.net At: Apr 18 2016 10:34:24
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?

On 04/18/2016 10:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
> Requiring users to remember to pass specific userdata through to their
> instance at every launch in order to replace functionality that
> currently works invisible to them would be a step backwards. It's an
> alternative, yes, but it's an alternative that adds burden to our users
> and is not one we would pursue.
> 
> What is the rationale for desiring to remove this functionality?

The Nova team would like to remove every config option that specifies an
arbitrary out of tree class file at a function point. This has been the
sentiment for a while and we did a wave of deprecations at the end of
Mitaka to signal this more broadly, because as an arbitrary class loader
it completely impossible to even understand who might be using it and how.

These interfaces are not considered stable or contractual, so exposing
them as raw class loader is something that we want to stop doing, as
we're going to horribly break people at some point. It's fine if there
are multiple implementations for these things, however those should all
be upstream, and selected by a symbolic name CONF option.

One of the alternatives is to propose your solution upstream.

  -Sean

-- 
Sean Dague
http://dague.net

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Sean Dague]

On 04/18/2016 10:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
> Requiring users to remember to pass specific userdata through to their
> instance at every launch in order to replace functionality that
> currently works invisible to them would be a step backwards. It's an
> alternative, yes, but it's an alternative that adds burden to our users
> and is not one we would pursue.
> 
> What is the rationale for desiring to remove this functionality?

The Nova team would like to remove every config option that specifies an
arbitrary out of tree class file at a function point. This has been the
sentiment for a while and we did a wave of deprecations at the end of
Mitaka to signal this more broadly, because as an arbitrary class loader
it completely impossible to even understand who might be using it and how.

These interfaces are not considered stable or contractual, so exposing
them as raw class loader is something that we want to stop doing, as
we're going to horribly break people at some point. It's fine if there
are multiple implementations for these things, however those should all
be upstream, and selected by a symbolic name CONF option.

One of the alternatives is to propose your solution upstream.

-Sean

-- 
Sean Dague
http://dague.net

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[David Medberry]

Hi Ned, Jay,

We use it also and I have to agree, it's onerous to require users to add
that functionality back in. Where was this discussed?

On Mon, Apr 18, 2016 at 8:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) <
erh...@bloomberg.net> wrote:

> Requiring users to remember to pass specific userdata through to their
> instance at every launch in order to replace functionality that currently
> works invisible to them would be a step backwards. It's an alternative,
> yes, but it's an alternative that adds burden to our users and is not one
> we would pursue.
>
> What is the rationale for desiring to remove this functionality?
>
> From: jaypi...@gmail.com
> Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in
> nova.conf?
>
> On 04/18/2016 09:24 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
> > I noticed while reading through Mitaka release notes that
> > vendordata_driver has been deprecated in Mitaka
> > (https://review.openstack.org/#/c/288107/) and is slated for removal at
> > some point. This came as somewhat of a surprise to me - I searched
> > openstack-dev for vendordata-related subject lines going back to January
> > and found no discussion on the matter (IRC logs, while available on
> > eavesdrop, are not trivially searchable without a little scripting to
> > fetch them first, so I didn't check there yet).
> >
> > We at Bloomberg make heavy use of this particular feature to inject
> > dynamically generated JSON into the metadata service of instances; the
> > content of the JSON differs depending on the instance making the request
> > to the metadata service. The functionality that adds the contents of a
> > static JSON file, while remaining around, is not suitable for our use
> case.
> >
> > Please let me know if you use vendordata_driver so that I/we can present
> > an organized case for why this option or equivalent functionality needs
> > to remain around. The alternative is that we end up patching the
> > vendordata driver directly in Nova when we move to Mitaka, which I'd
> > like to avoid; as a matter of principle I would rather see more
> > classloader overrides, not fewer.
>
> Wouldn't an alternative be to use something like Chef, Puppet, Ansible,
> Saltstack, etc and their associated config variable storage services
> like Hiera or something similar to publish custom metadata? That way,
> all you need to pass to your instance (via userdata) is a URI or
> connection string and some auth details for your config storage service
> and the instance can grab whatever you need.
>
> Thoughts?
> -jay
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[David Medberry]

Ah, read the bug, we are using the default, not a custom driver. so
NEVERMIND.

On Mon, Apr 18, 2016 at 8:16 AM, David Medberry <openst...@medberry.net>
wrote:

> Hi Ned, Jay,
>
> We use it also and I have to agree, it's onerous to require users to add
> that functionality back in. Where was this discussed?
>
> On Mon, Apr 18, 2016 at 8:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) <
> erh...@bloomberg.net> wrote:
>
>> Requiring users to remember to pass specific userdata through to their
>> instance at every launch in order to replace functionality that currently
>> works invisible to them would be a step backwards. It's an alternative,
>> yes, but it's an alternative that adds burden to our users and is not one
>> we would pursue.
>>
>> What is the rationale for desiring to remove this functionality?
>>
>> From: jaypi...@gmail.com
>> Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in
>> nova.conf?
>>
>> On 04/18/2016 09:24 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
>> > I noticed while reading through Mitaka release notes that
>> > vendordata_driver has been deprecated in Mitaka
>> > (https://review.openstack.org/#/c/288107/) and is slated for removal at
>> > some point. This came as somewhat of a surprise to me - I searched
>> > openstack-dev for vendordata-related subject lines going back to January
>> > and found no discussion on the matter (IRC logs, while available on
>> > eavesdrop, are not trivially searchable without a little scripting to
>> > fetch them first, so I didn't check there yet).
>> >
>> > We at Bloomberg make heavy use of this particular feature to inject
>> > dynamically generated JSON into the metadata service of instances; the
>> > content of the JSON differs depending on the instance making the request
>> > to the metadata service. The functionality that adds the contents of a
>> > static JSON file, while remaining around, is not suitable for our use
>> case.
>> >
>> > Please let me know if you use vendordata_driver so that I/we can present
>> > an organized case for why this option or equivalent functionality needs
>> > to remain around. The alternative is that we end up patching the
>> > vendordata driver directly in Nova when we move to Mitaka, which I'd
>> > like to avoid; as a matter of principle I would rather see more
>> > classloader overrides, not fewer.
>>
>> Wouldn't an alternative be to use something like Chef, Puppet, Ansible,
>> Saltstack, etc and their associated config variable storage services
>> like Hiera or something similar to publish custom metadata? That way,
>> all you need to pass to your instance (via userdata) is a URI or
>> connection string and some auth details for your config storage service
>> and the instance can grab whatever you need.
>>
>> Thoughts?
>> -jay
>>
>> ___
>> OpenStack-operators mailing list
>> OpenStack-operators@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>>
>>
>> ___
>> OpenStack-operators mailing list
>> OpenStack-operators@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>>
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Ned Rhudy (BLOOMBERG/ 731 LEX)]

Requiring users to remember to pass specific userdata through to their instance 
at every launch in order to replace functionality that currently works 
invisible to them would be a step backwards. It's an alternative, yes, but it's 
an alternative that adds burden to our users and is not one we would pursue.

What is the rationale for desiring to remove this functionality?

From: jaypi...@gmail.com 
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?

On 04/18/2016 09:24 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
> I noticed while reading through Mitaka release notes that
> vendordata_driver has been deprecated in Mitaka
> (https://review.openstack.org/#/c/288107/) and is slated for removal at
> some point. This came as somewhat of a surprise to me - I searched
> openstack-dev for vendordata-related subject lines going back to January
> and found no discussion on the matter (IRC logs, while available on
> eavesdrop, are not trivially searchable without a little scripting to
> fetch them first, so I didn't check there yet).
>
> We at Bloomberg make heavy use of this particular feature to inject
> dynamically generated JSON into the metadata service of instances; the
> content of the JSON differs depending on the instance making the request
> to the metadata service. The functionality that adds the contents of a
> static JSON file, while remaining around, is not suitable for our use case.
>
> Please let me know if you use vendordata_driver so that I/we can present
> an organized case for why this option or equivalent functionality needs
> to remain around. The alternative is that we end up patching the
> vendordata driver directly in Nova when we move to Mitaka, which I'd
> like to avoid; as a matter of principle I would rather see more
> classloader overrides, not fewer.

Wouldn't an alternative be to use something like Chef, Puppet, Ansible, 
Saltstack, etc and their associated config variable storage services 
like Hiera or something similar to publish custom metadata? That way, 
all you need to pass to your instance (via userdata) is a URI or 
connection string and some auth details for your config storage service 
and the instance can grab whatever you need.

Thoughts?
-jay

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Jay Pipes]

On 04/18/2016 09:24 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:

I noticed while reading through Mitaka release notes that
vendordata_driver has been deprecated in Mitaka
(https://review.openstack.org/#/c/288107/) and is slated for removal at
some point. This came as somewhat of a surprise to me - I searched
openstack-dev for vendordata-related subject lines going back to January
and found no discussion on the matter (IRC logs, while available on
eavesdrop, are not trivially searchable without a little scripting to
fetch them first, so I didn't check there yet).

We at Bloomberg make heavy use of this particular feature to inject
dynamically generated JSON into the metadata service of instances; the
content of the JSON differs depending on the instance making the request
to the metadata service. The functionality that adds the contents of a
static JSON file, while remaining around, is not suitable for our use case.

Please let me know if you use vendordata_driver so that I/we can present
an organized case for why this option or equivalent functionality needs
to remain around. The alternative is that we end up patching the
vendordata driver directly in Nova when we move to Mitaka, which I'd
like to avoid; as a matter of principle I would rather see more
classloader overrides, not fewer.


Wouldn't an alternative be to use something like Chef, Puppet, Ansible, 
Saltstack, etc and their associated config variable storage services 
like Hiera or something similar to publish custom metadata? That way, 
all you need to pass to your instance (via userdata) is a URI or 
connection string and some auth details for your config storage service 
and the instance can grab whatever you need.


Thoughts?
-jay

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Ned Rhudy (BLOOMBERG/ 731 LEX)]

I noticed while reading through Mitaka release notes that vendordata_driver has 
been deprecated in Mitaka (https://review.openstack.org/#/c/288107/) and is 
slated for removal at some point. This came as somewhat of a surprise to me - I 
searched openstack-dev for vendordata-related subject lines going back to 
January and found no discussion on the matter (IRC logs, while available on 
eavesdrop, are not trivially searchable without a little scripting to fetch 
them first, so I didn't check there yet).

We at Bloomberg make heavy use of this particular feature to inject dynamically 
generated JSON into the metadata service of instances; the content of the JSON 
differs depending on the instance making the request to the metadata service. 
The functionality that adds the contents of a static JSON file, while remaining 
around, is not suitable for our use case.

Please let me know if you use vendordata_driver so that I/we can present an 
organized case for why this option or equivalent functionality needs to remain 
around. The alternative is that we end up patching the vendordata driver 
directly in Nova when we move to Mitaka, which I'd like to avoid; as a matter 
of principle I would rather see more classloader overrides, not fewer.___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators