Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Ed Leafe]

On 03/03/2016 09:09 AM, Sam Matzek wrote:

>> > So, don't deprecate until you have a solution.  All you will be doing is
>> > putting people in a tight spot where they will have to fork the code base,
>> > and that is downright antisocial.
>> >
>> > Let's plan this out in the Newton Summit and have a plan moving forward.
> Deprecate isn't the same as remove unless I'm missing something on how
> this works.  I think we want to deprecate it to discourage further
> use, to gather current use cases, and to drive approved specs for
> those use cases.   Hooks should not be removed from tree until we have
> the replacements in tree.

Deprecate ideally means "Don't use this anymore, as it is not the
recommended approach, and will not be supported/available in the future.
Instead, use XXX."

In other words, don't tell people not to use something unless you can
point them to a better way to accomplish what they need to do.

-- 

-- Ed Leafe

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Daniel P. Berrange]

On Thu, Mar 03, 2016 at 09:09:03AM -0600, Sam Matzek wrote:
> On Wed, Mar 2, 2016 at 8:25 PM, Adam Young  wrote:
> > On 02/29/2016 01:49 PM, Andrew Laski wrote:
> >>
> >>
> >> On Mon, Feb 29, 2016, at 01:18 PM, Dan Smith wrote:
> 
>  Forgive my ignorance or for playing devil's advocate, but wouldn't the
>  main difference between notifications and hooks be that notifications
>  are asynchronous and hooks aren't?
> >>>
> >>> The main difference is that notifications are external and intended to
> >>> be stable (especially with the versioned notifications effort). The
> >>> hooks are internal and depend wholly on internal data structures.
> >>>
>  In the case of how Rdo was using it,
>  they are adding things to the injected_files list before the instance is
>  created in the compute API.  You couldn't do that with notifications as
>  far as I know.
> >>>
> >>> Nope, definitely not, but I see that as a good thing. Injecting files
> >>> like that is likely to be very fragile and I think mostly regarded as
> >>> substantially less desirable than the alternatives, regardless of how it
> >>> happens.
> >>>
> >>> I think that Laski's point was that the most useful and least dangerous
> >>> thing that hooks can be used for is the use case that is much better
> >>> served by notifications.
> >
> >
> > I did the original proof-of-concept for this prior to the impl using the
> > hooks, just by modifying the metadata.
> >
> > http://adam.younglogic.com/2013/09/register-vm-freeipa/
> >
> > That works for a CLI based approach, but not for auto-registering VMs
> > created from the WebUI, and also only works if the user crafts the Metadata
> > properly.  It was not a secure approach.
> >
> > What we need is a way to be able to generate a secret and share that between
> > the new VM and the enrolling server.  The call does not strictly have to be
> > synchronous.  The enrolling server can wait if the VM is not up, and the VM
> > can wait if the enrolling server does not have the secret when the VM is
> > ready to enroll.
> >
> > We had discussed having a seperate service listen to notifications on the
> > bus and inject the data necessary into the IdM server.  The hooks were a
> > much better solution.
> >
> > I had seriously thought about using the Keystone token as the symmetric
> > shared secret.  It is a horrible solution, but so are all the rest.
> >
> > There is no security on the message bus at the moment, and until we get
> > some, we can't put a secret on the bus.
> >
> > So, don't deprecate until you have a solution.  All you will be doing is
> > putting people in a tight spot where they will have to fork the code base,
> > and that is downright antisocial.
> >
> > Let's plan this out in the Newton Summit and have a plan moving forward.
> 
> Deprecate isn't the same as remove unless I'm missing something on how
> this works.  I think we want to deprecate it to discourage further
> use, to gather current use cases, and to drive approved specs for
> those use cases.   Hooks should not be removed from tree until we have
> the replacements in tree.

Yes & no. We're not committing to supporting every single thing people
do via hooks today. We're saying that we want people to use notifications
primarily as a way to trigger async execution of hooks outside of nova.
Notifications already exist, but we're lacking an official way to trigger
scripts based off the receipt of a notification. So once that's done we
will have an official replacement available, and we'll consider removing
the hooks mechanism at this point. This will certainly /not/ cover all
possible use cases that hooks are used for today, and that is by design,
since we're explicitly not wanting people to be able to arbitrarily
change nova functionality.

We will however consider requests to add more data fields to notifications
if some key info is missing. Also if there are things that can't be done
via async notifications, we'll consider specs to add explicit features to
achieve a similar goal in nova. That doesn't mean we're going to accept
every request, sumply that we will evaluate them on merit, as we would
with any other Nova feature. Whether we wait for all these potential
feature requests to be implemented before we remove hooks is an open
question. I think it'll depend on the scope of the proposed features and
their importance to the Nova ecosystem, vs other features we have to deal
with implementing. IOW this will be a case by case decision, and we're
not going to wait an abitrarily long time for people to propose replaement
features before deleting hooks.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Sam Matzek]

On Wed, Mar 2, 2016 at 8:25 PM, Adam Young  wrote:
> On 02/29/2016 01:49 PM, Andrew Laski wrote:
>>
>>
>> On Mon, Feb 29, 2016, at 01:18 PM, Dan Smith wrote:

 Forgive my ignorance or for playing devil's advocate, but wouldn't the
 main difference between notifications and hooks be that notifications
 are asynchronous and hooks aren't?
>>>
>>> The main difference is that notifications are external and intended to
>>> be stable (especially with the versioned notifications effort). The
>>> hooks are internal and depend wholly on internal data structures.
>>>
 In the case of how Rdo was using it,
 they are adding things to the injected_files list before the instance is
 created in the compute API.  You couldn't do that with notifications as
 far as I know.
>>>
>>> Nope, definitely not, but I see that as a good thing. Injecting files
>>> like that is likely to be very fragile and I think mostly regarded as
>>> substantially less desirable than the alternatives, regardless of how it
>>> happens.
>>>
>>> I think that Laski's point was that the most useful and least dangerous
>>> thing that hooks can be used for is the use case that is much better
>>> served by notifications.
>
>
> I did the original proof-of-concept for this prior to the impl using the
> hooks, just by modifying the metadata.
>
> http://adam.younglogic.com/2013/09/register-vm-freeipa/
>
> That works for a CLI based approach, but not for auto-registering VMs
> created from the WebUI, and also only works if the user crafts the Metadata
> properly.  It was not a secure approach.
>
> What we need is a way to be able to generate a secret and share that between
> the new VM and the enrolling server.  The call does not strictly have to be
> synchronous.  The enrolling server can wait if the VM is not up, and the VM
> can wait if the enrolling server does not have the secret when the VM is
> ready to enroll.
>
> We had discussed having a seperate service listen to notifications on the
> bus and inject the data necessary into the IdM server.  The hooks were a
> much better solution.
>
> I had seriously thought about using the Keystone token as the symmetric
> shared secret.  It is a horrible solution, but so are all the rest.
>
> There is no security on the message bus at the moment, and until we get
> some, we can't put a secret on the bus.
>
> So, don't deprecate until you have a solution.  All you will be doing is
> putting people in a tight spot where they will have to fork the code base,
> and that is downright antisocial.
>
> Let's plan this out in the Newton Summit and have a plan moving forward.

Deprecate isn't the same as remove unless I'm missing something on how
this works.  I think we want to deprecate it to discourage further
use, to gather current use cases, and to drive approved specs for
those use cases.   Hooks should not be removed from tree until we have
the replacements in tree.

One other way to do this that I haven't seen mentioned in the thread
yet, is create your own middleware code and put it in the wgsi
pipeline in Nova api-paste.  For the shared secret / injected file
case you could process the create server request after keystone but
before Nova API proper, do your synchronous call outs, and then add
your files to the injected files part of the Nova request body.

>
>
>
>
>> Yep. My experience with them was things like updating an external cache
>> on create/delete or calling out to a DNS provider to remove a reverse
>> DNS entry on instance delete. Things that could easily be handled with
>> notifications, and use cases that I think we should continue to support
>> by improving notifications if necessary.
>>
>>
>>> So, if file injection (and any other internals-mangling that other
>>> people may use them for) is not a reason to keep hooks, and if
>>> notifications are the proper way to trigger on things happening, then
>>> there's no reason to keep hooks.
>>>
>>> --Dan
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Adam Young]

On 02/29/2016 01:49 PM, Andrew Laski wrote:


On Mon, Feb 29, 2016, at 01:18 PM, Dan Smith wrote:

Forgive my ignorance or for playing devil's advocate, but wouldn't the
main difference between notifications and hooks be that notifications
are asynchronous and hooks aren't?

The main difference is that notifications are external and intended to
be stable (especially with the versioned notifications effort). The
hooks are internal and depend wholly on internal data structures.


In the case of how Rdo was using it,
they are adding things to the injected_files list before the instance is
created in the compute API.  You couldn't do that with notifications as
far as I know.

Nope, definitely not, but I see that as a good thing. Injecting files
like that is likely to be very fragile and I think mostly regarded as
substantially less desirable than the alternatives, regardless of how it
happens.

I think that Laski's point was that the most useful and least dangerous
thing that hooks can be used for is the use case that is much better
served by notifications.


I did the original proof-of-concept for this prior to the impl using the 
hooks, just by modifying the metadata.


http://adam.younglogic.com/2013/09/register-vm-freeipa/

That works for a CLI based approach, but not for auto-registering VMs 
created from the WebUI, and also only works if the user crafts the 
Metadata properly.  It was not a secure approach.


What we need is a way to be able to generate a secret and share that 
between the new VM and the enrolling server.  The call does not strictly 
have to be synchronous.  The enrolling server can wait if the VM is not 
up, and the VM can wait if the enrolling server does not have the secret 
when the VM is ready to enroll.


We had discussed having a seperate service listen to notifications on 
the bus and inject the data necessary into the IdM server.  The hooks 
were a much better solution.


I had seriously thought about using the Keystone token as the symmetric 
shared secret.  It is a horrible solution, but so are all the rest.


There is no security on the message bus at the moment, and until we get 
some, we can't put a secret on the bus.


So, don't deprecate until you have a solution.  All you will be doing is 
putting people in a tight spot where they will have to fork the code 
base, and that is downright antisocial.


Let's plan this out in the Newton Summit and have a plan moving forward.




Yep. My experience with them was things like updating an external cache
on create/delete or calling out to a DNS provider to remove a reverse
DNS entry on instance delete. Things that could easily be handled with
notifications, and use cases that I think we should continue to support
by improving notifications if necessary.



So, if file injection (and any other internals-mangling that other
people may use them for) is not a reason to keep hooks, and if
notifications are the proper way to trigger on things happening, then
there's no reason to keep hooks.

--Dan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Sean Dague]

On 03/02/2016 03:01 AM, Juan Antonio Osorio wrote:
> 
> 
> On Tue, Mar 1, 2016 at 2:14 PM, Sean Dague  > wrote:
> 
> On 02/29/2016 05:23 PM, Matt Riedemann wrote:
> >
> >
> > On 2/29/2016 2:54 PM, Sean Dague wrote:
> >> On 02/29/2016 11:59 AM, Sean Dague wrote:
> >>> The nova/hooks.py infrastructure has been with us since early
> Nova. It's
> >>> currently only annotated on a few locations - 'build_instance',
> >>> 'create_instance', 'delete_instance', and
> 'instance_network_info'. It's
> >>> got a couple of unit tests on it, but nothing that actually
> tests real
> >>> behavior of the hooks we have specified.
> >>>
> >>> It does get used in the wild, and we do break it with changes we
> didn't
> >>> ever anticipate would impact it -
> >>> https://bugs.launchpad.net/nova/+bug/1518321
> >>>
> >>> However, when you look into how that is used, it's really really
> odd and
> >>> fragile -
> >>>
> 
> https://github.com/richm/rdo-vm-factory/blob/master/rdo-ipa-nova/novahooks.py#L248
> >>>
> >>>
> >>>
> >>>  def pre(self, *args, **kwargs):
> >>>  # args[7] is the injected_files parameter array
> >>>  # the value is ('filename', 'base64 encoded contents')
> >>>  ipaotp = str(uuid.uuid4())
> >>>  ipainject = ('/tmp/ipaotp', base64.b64encode(ipaotp))
> >>>  args[7].extend(self.inject_files)
> >>>  args[7].append(ipainject)
> >>>
> >>> In our continued quest on being more explicit about plug points
> it feels
> >>> like we should other document the interface (which means creating
> >>> stability on the hook parameters) or we should deprecate this
> construct
> >>> as part of a bygone era.
> >>>
> >>> I lean on deprecation because it feels like a thing we don't
> really want
> >>> to support going forward, but I can go either way.
> >>>
> >>> -Sean
> >>>
> >>> P.S. I'm starting to look at in tree functional testing for all
> of this,
> >>> in the event that we decide not to deprecate it. It's definitely
> made a
> >>> little hard by the way all exceptions are caught when hooks go
> wrong.
> >>
> >> As there seemed to be some early enthusiasm for this from the
> core team,
> >> the deprecation patch is proposed here -
> >> https://review.openstack.org/#/c/286276/
> >>
> >> Unless there is a big reversal of sentiment I'd suggest we get that
> >> landed in Mitaka so that this is signalled to people going
> forward, and
> >> we can collect use cases for things that are currently using
> hooks that
> >> we may want to support in other ways in Newton.
> >>
> >> -Sean
> >>
> >
> > I think you should probably also post this to the operators
> mailing list
> > before we actually deprecate hooks.
> 
> I'm not sure that's really true. As a team we have to decide we're going
> to deprecate or document this facility. The status quo is really not a
> thing.
> 
> 
> I'm not sure what you mean by status quo, it's just asking the actual
> users of
> the software what they think and if this will affect their implementations.
>  
> 
> 
> So unless we've got folks that believe we should crystalize this
> interface, and are signing up for doing that (which means you can't
> change build_instance internal function parameters ever), then I think
> it's fine to deprecate. We can measure our removal timeframe based on
> reaction to that.
> 
> 
> So then, would it be acceptable to come up with a blueprint trying to
> formalize
> the interfaces for the hooks, and make those standard (and better tested and
> documented)? We could base the deprecation on the outcome of that blueprint.
> 
> I think everybody agrees that the current solution is not ideal, but I
> think it covers
> a valid use-case.

I think the sentiment is pretty clear that formalizing hooks is not the
right way forward. The specific feature you are looking for should come
in via a nova-spec.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Daniel P. Berrange]

On Tue, Mar 01, 2016 at 01:53:18PM -0500, Rob Crittenden wrote:
> Daniel P. Berrange wrote:
> > On Mon, Feb 29, 2016 at 12:36:03PM -0700, Rich Megginson wrote:
> >> On 02/29/2016 12:19 PM, Chris Friesen wrote:
> >>> On 02/29/2016 12:22 PM, Daniel P. Berrange wrote:
> >>>
>  There's three core scenarios for hooks
> 
>   1. Modifying some aspect of the Nova operation
>   2. Triggering an external action synchronously to some Nova operation
>   3. Triggering an external action asynchronously to some Nova operation
> 
>  The Rdo example is falling in scenario 1 since it is modifying the
>  injected files. I think this is is absolutely the kind of thing
>  we should explicitly *never* support. When external code can arbitrarily
>  modify some aspect of Nova operation we're in totally unchartered
>  territory as to the behaviour of Nova. To support that we'd need to
>  provide a stable internal API which is just not something we want to
>  tie ourselves into. I don't know just what the Rdo example is trying
>  to achieve, but whatever it is, it should be via some supportable API
>  and not a hook.,
> 
>  Scenaris 2 and 3 are both valid to consider. Using the notifications
>  system gets as an asynchronous trigger mechanism, which is probably
>  fine for many scenarios.  The big question is whether there's a
>  compelling need for scenario two, where the external action blocks
>  execution of the Nova operation until it has completed its hook.
> >>>
> >>> Even in the case of scenario two it is possible in some cases to use a
> >>> proxy to intercept the HTTP request, take action, and then forward it or
> >>> reject it as appropriate.
> >>>
> >>> I think the real question is whether there's a need to trigger an external
> >>> action synchronously from down in the guts of the nova code.
> >>
> >> The hooks do the following: 
> >> https://github.com/rcritten/rdo-vm-factory/blob/use-centos/rdo-ipa-nova/novahooks.py#L271
> >>
> >> We need to generate a token (ipaotp) and call ipa host-add with that token
> >> _before_ the new machine has a chance to call ipa-client-install.  We have
> >> to guarantee that the client cannot call ipa-client-install until we get
> >> back the response from ipa that the host has been added with the token.
> >> Additionally, we store the token in an injected_file in the new machine, so
> >> the file can be removed as soon as possible.  We tried storing the token in
> >> the VM metadata, but there is apparently no way to delete it?  Can the
> >> machine do
> >>
> >> curl -XDELETE http://168.254.169.254/openstack/latest/metadata?key=value ?
> >>
> >> Using the build_instance.pre hook in Nova makes this simple and
> >> straightforward.  It's also relatively painless to use the 
> >> network_info.post
> >> hook to handle the floating ip address assignment.
> >>
> >> Is it possible to do the above using notifications without jumping through
> >> too many hoops?
> > 
> > So from a high level POV, you are trying to generate a security token
> > which will be provided to the guest OS before it is booted.
> > 
> > I think that is a pretty clearly useful feature, and something that
> > should really be officially integrated into Nova as a concept rather
> > than done behind nova's back as a hook.
> 
> Note that the reason the file was injected the way it was is so that
> Nova would have no idea there even is a token. We didn't want someone
> later peeking at the metadata, or a notification, to get the token.

That's just security through obscurity

Anyway, if you wish to be able to continue to support your use case,
I'll strongly recommend that you look to propose a mechanism to officially
support provision of tokens to guest OS in Nova. It is pretty clear from
this thread that the hooks mechanism you rely on is going to be deleted
in the future, likely as soon as the Newton release.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Juan Antonio Osorio]

On Tue, Mar 1, 2016 at 2:14 PM, Sean Dague  wrote:

> On 02/29/2016 05:23 PM, Matt Riedemann wrote:
> >
> >
> > On 2/29/2016 2:54 PM, Sean Dague wrote:
> >> On 02/29/2016 11:59 AM, Sean Dague wrote:
> >>> The nova/hooks.py infrastructure has been with us since early Nova.
> It's
> >>> currently only annotated on a few locations - 'build_instance',
> >>> 'create_instance', 'delete_instance', and 'instance_network_info'. It's
> >>> got a couple of unit tests on it, but nothing that actually tests real
> >>> behavior of the hooks we have specified.
> >>>
> >>> It does get used in the wild, and we do break it with changes we didn't
> >>> ever anticipate would impact it -
> >>> https://bugs.launchpad.net/nova/+bug/1518321
> >>>
> >>> However, when you look into how that is used, it's really really odd
> and
> >>> fragile -
> >>>
> https://github.com/richm/rdo-vm-factory/blob/master/rdo-ipa-nova/novahooks.py#L248
> >>>
> >>>
> >>>
> >>>  def pre(self, *args, **kwargs):
> >>>  # args[7] is the injected_files parameter array
> >>>  # the value is ('filename', 'base64 encoded contents')
> >>>  ipaotp = str(uuid.uuid4())
> >>>  ipainject = ('/tmp/ipaotp', base64.b64encode(ipaotp))
> >>>  args[7].extend(self.inject_files)
> >>>  args[7].append(ipainject)
> >>>
> >>> In our continued quest on being more explicit about plug points it
> feels
> >>> like we should other document the interface (which means creating
> >>> stability on the hook parameters) or we should deprecate this construct
> >>> as part of a bygone era.
> >>>
> >>> I lean on deprecation because it feels like a thing we don't really
> want
> >>> to support going forward, but I can go either way.
> >>>
> >>> -Sean
> >>>
> >>> P.S. I'm starting to look at in tree functional testing for all of
> this,
> >>> in the event that we decide not to deprecate it. It's definitely made a
> >>> little hard by the way all exceptions are caught when hooks go wrong.
> >>
> >> As there seemed to be some early enthusiasm for this from the core team,
> >> the deprecation patch is proposed here -
> >> https://review.openstack.org/#/c/286276/
> >>
> >> Unless there is a big reversal of sentiment I'd suggest we get that
> >> landed in Mitaka so that this is signalled to people going forward, and
> >> we can collect use cases for things that are currently using hooks that
> >> we may want to support in other ways in Newton.
> >>
> >> -Sean
> >>
> >
> > I think you should probably also post this to the operators mailing list
> > before we actually deprecate hooks.
>
> I'm not sure that's really true. As a team we have to decide we're going
> to deprecate or document this facility. The status quo is really not a
> thing.
>

I'm not sure what you mean by status quo, it's just asking the actual users
of
the software what they think and if this will affect their implementations.


>
> So unless we've got folks that believe we should crystalize this
> interface, and are signing up for doing that (which means you can't
> change build_instance internal function parameters ever), then I think
> it's fine to deprecate. We can measure our removal timeframe based on
> reaction to that.
>

So then, would it be acceptable to come up with a blueprint trying to
formalize
the interfaces for the hooks, and make those standard (and better tested and
documented)? We could base the deprecation on the outcome of that blueprint.

I think everybody agrees that the current solution is not ideal, but I
think it covers
a valid use-case.


>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Juan Antonio Osorio R.
e-mail: jaosor...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Sean Dague]

On 03/01/2016 02:04 PM, Rob Crittenden wrote:
> Daniel P. Berrange wrote:
>> On Mon, Feb 29, 2016 at 11:59:06AM -0500, Sean Dague wrote:
>>> The nova/hooks.py infrastructure has been with us since early Nova. It's
>>> currently only annotated on a few locations - 'build_instance',
>>> 'create_instance', 'delete_instance', and 'instance_network_info'. It's
>>> got a couple of unit tests on it, but nothing that actually tests real
>>> behavior of the hooks we have specified.
>>>
>>> It does get used in the wild, and we do break it with changes we didn't
>>> ever anticipate would impact it -
>>> https://bugs.launchpad.net/nova/+bug/1518321
>>>
>>> However, when you look into how that is used, it's really really odd and
>>> fragile -
>>> https://github.com/richm/rdo-vm-factory/blob/master/rdo-ipa-nova/novahooks.py#L248
>>>
>>>
>>> def pre(self, *args, **kwargs):
>>> # args[7] is the injected_files parameter array
>>> # the value is ('filename', 'base64 encoded contents')
>>> ipaotp = str(uuid.uuid4())
>>> ipainject = ('/tmp/ipaotp', base64.b64encode(ipaotp))
>>> args[7].extend(self.inject_files)
>>> args[7].append(ipainject)
>>>
>>> In our continued quest on being more explicit about plug points it feels
>>> like we should other document the interface (which means creating
>>> stability on the hook parameters) or we should deprecate this construct
>>> as part of a bygone era.
>>>
>>> I lean on deprecation because it feels like a thing we don't really want
>>> to support going forward, but I can go either way.
>>
>> As it is designed, I think the hooks mechanism is really unsupportable
>> long term. It is exposing users to arbitrary internal Nova data structures
>> which we have changed at will and we cannot possibly ever consider them
>> to be a stable consumable API. I'm rather surprised we've not seen more
>> bugs like the one you've shown above - most likely thats a reflection
>> on not many people actually using this facility.
>>
>> I'd be strongly in favour of deprecation & removal of this hooking
>> mechanism, as its unsupportable in any sane manner when it exposes
>> code to our internal unstable API & object model.
>>
>> If there's stuff people are doing in hooks that's impossible any other
>> way, we should really be looking at what we need to provide in our
>> public API to achieve the same goal, if it is use case we wish to be
>> able to support.
> 
> I'll just add that lots and lots of software has hooks. Just because the
> initial implementation decided to expose internal structures (which lots
> of other APIs do in order to be interesting) doesn't make it inherently
> bad. It just requires a stable API rather than passing *args, **kwargs
> around. It seems like the original design was to throw in the kitchen
> sink when a more targeted set of arguments would probably do just fine.

The original implementation comes from a time when OpenStack was a kit
to build cloud software out of. And nearly every bit was extendable in
various ways.

We are now transitioning into a phase where interoperability is more
important than differentiation. That means deprecating out some
generalized hook points and requiring that features be baked in, so that
all clouds will have them, and user experience on different things that
are OpenStack (tm) are more similar.

At the end of the day this is python, so you can monkey patch anything
you want. But by documenting a hook point we endorse that. And the
sentiment in this thread is pretty much that we can't endorse this one
any more.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Rob Crittenden]

Daniel P. Berrange wrote:
> On Mon, Feb 29, 2016 at 11:59:06AM -0500, Sean Dague wrote:
>> The nova/hooks.py infrastructure has been with us since early Nova. It's
>> currently only annotated on a few locations - 'build_instance',
>> 'create_instance', 'delete_instance', and 'instance_network_info'. It's
>> got a couple of unit tests on it, but nothing that actually tests real
>> behavior of the hooks we have specified.
>>
>> It does get used in the wild, and we do break it with changes we didn't
>> ever anticipate would impact it -
>> https://bugs.launchpad.net/nova/+bug/1518321
>>
>> However, when you look into how that is used, it's really really odd and
>> fragile -
>> https://github.com/richm/rdo-vm-factory/blob/master/rdo-ipa-nova/novahooks.py#L248
>>
>>
>> def pre(self, *args, **kwargs):
>> # args[7] is the injected_files parameter array
>> # the value is ('filename', 'base64 encoded contents')
>> ipaotp = str(uuid.uuid4())
>> ipainject = ('/tmp/ipaotp', base64.b64encode(ipaotp))
>> args[7].extend(self.inject_files)
>> args[7].append(ipainject)
>>
>> In our continued quest on being more explicit about plug points it feels
>> like we should other document the interface (which means creating
>> stability on the hook parameters) or we should deprecate this construct
>> as part of a bygone era.
>>
>> I lean on deprecation because it feels like a thing we don't really want
>> to support going forward, but I can go either way.
> 
> As it is designed, I think the hooks mechanism is really unsupportable
> long term. It is exposing users to arbitrary internal Nova data structures
> which we have changed at will and we cannot possibly ever consider them
> to be a stable consumable API. I'm rather surprised we've not seen more
> bugs like the one you've shown above - most likely thats a reflection
> on not many people actually using this facility.
> 
> I'd be strongly in favour of deprecation & removal of this hooking
> mechanism, as its unsupportable in any sane manner when it exposes
> code to our internal unstable API & object model.
> 
> If there's stuff people are doing in hooks that's impossible any other
> way, we should really be looking at what we need to provide in our
> public API to achieve the same goal, if it is use case we wish to be
> able to support.

I'll just add that lots and lots of software has hooks. Just because the
initial implementation decided to expose internal structures (which lots
of other APIs do in order to be interesting) doesn't make it inherently
bad. It just requires a stable API rather than passing *args, **kwargs
around. It seems like the original design was to throw in the kitchen
sink when a more targeted set of arguments would probably do just fine.

rob

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Rob Crittenden]

Daniel P. Berrange wrote:
> On Mon, Feb 29, 2016 at 12:36:03PM -0700, Rich Megginson wrote:
>> On 02/29/2016 12:19 PM, Chris Friesen wrote:
>>> On 02/29/2016 12:22 PM, Daniel P. Berrange wrote:
>>>
 There's three core scenarios for hooks

  1. Modifying some aspect of the Nova operation
  2. Triggering an external action synchronously to some Nova operation
  3. Triggering an external action asynchronously to some Nova operation

 The Rdo example is falling in scenario 1 since it is modifying the
 injected files. I think this is is absolutely the kind of thing
 we should explicitly *never* support. When external code can arbitrarily
 modify some aspect of Nova operation we're in totally unchartered
 territory as to the behaviour of Nova. To support that we'd need to
 provide a stable internal API which is just not something we want to
 tie ourselves into. I don't know just what the Rdo example is trying
 to achieve, but whatever it is, it should be via some supportable API
 and not a hook.,

 Scenaris 2 and 3 are both valid to consider. Using the notifications
 system gets as an asynchronous trigger mechanism, which is probably
 fine for many scenarios.  The big question is whether there's a
 compelling need for scenario two, where the external action blocks
 execution of the Nova operation until it has completed its hook.
>>>
>>> Even in the case of scenario two it is possible in some cases to use a
>>> proxy to intercept the HTTP request, take action, and then forward it or
>>> reject it as appropriate.
>>>
>>> I think the real question is whether there's a need to trigger an external
>>> action synchronously from down in the guts of the nova code.
>>
>> The hooks do the following: 
>> https://github.com/rcritten/rdo-vm-factory/blob/use-centos/rdo-ipa-nova/novahooks.py#L271
>>
>> We need to generate a token (ipaotp) and call ipa host-add with that token
>> _before_ the new machine has a chance to call ipa-client-install.  We have
>> to guarantee that the client cannot call ipa-client-install until we get
>> back the response from ipa that the host has been added with the token.
>> Additionally, we store the token in an injected_file in the new machine, so
>> the file can be removed as soon as possible.  We tried storing the token in
>> the VM metadata, but there is apparently no way to delete it?  Can the
>> machine do
>>
>> curl -XDELETE http://168.254.169.254/openstack/latest/metadata?key=value ?
>>
>> Using the build_instance.pre hook in Nova makes this simple and
>> straightforward.  It's also relatively painless to use the network_info.post
>> hook to handle the floating ip address assignment.
>>
>> Is it possible to do the above using notifications without jumping through
>> too many hoops?
> 
> So from a high level POV, you are trying to generate a security token
> which will be provided to the guest OS before it is booted.
> 
> I think that is a pretty clearly useful feature, and something that
> should really be officially integrated into Nova as a concept rather
> than done behind nova's back as a hook.

Note that the reason the file was injected the way it was is so that
Nova would have no idea there even is a token. We didn't want someone
later peeking at the metadata, or a notification, to get the token.

rob


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Joshua Harlow]

Sorry for the top posting, but seems like doing so is ok,

So an idea, feel free to use it or not...

But if a in-memory notification mechanism is desired then a library that 
I have been using and extracted from taskflow could be helpful here,


It doesn't bring in the full oslo.messaging notifications (since its 
only for in-memory after-all); but it could be just the thing needed to 
hook into observation points (with a more concrete definition/schema of 
what is/can be being observed being done by the nova folks):


https://notifier.readthedocs.org

https://pypi.python.org/pypi/notifier/

A really simple example:

>>> import notifier
>>> import time
>>> def watcher(event_type, details):
...print("%s happened with details: %s" % (event_type, details))
...
>>> n = notifier.Notifier()
>>> n.register('vm-booted', watcher)
0x7f1e6f294c80>'>

>>> n.notify('vm-booted', details={'when': time.time()})
vm-booted happened with details: {'when': 1456855953.650803}


Already in global-requirements btw,

-Josh

On 02/29/2016 08:59 AM, Sean Dague wrote:

The nova/hooks.py infrastructure has been with us since early Nova. It's
currently only annotated on a few locations - 'build_instance',
'create_instance', 'delete_instance', and 'instance_network_info'. It's
got a couple of unit tests on it, but nothing that actually tests real
behavior of the hooks we have specified.

It does get used in the wild, and we do break it with changes we didn't
ever anticipate would impact it -
https://bugs.launchpad.net/nova/+bug/1518321

However, when you look into how that is used, it's really really odd and
fragile -
https://github.com/richm/rdo-vm-factory/blob/master/rdo-ipa-nova/novahooks.py#L248


 def pre(self, *args, **kwargs):
 # args[7] is the injected_files parameter array
 # the value is ('filename', 'base64 encoded contents')
 ipaotp = str(uuid.uuid4())
 ipainject = ('/tmp/ipaotp', base64.b64encode(ipaotp))
 args[7].extend(self.inject_files)
 args[7].append(ipainject)

In our continued quest on being more explicit about plug points it feels
like we should other document the interface (which means creating
stability on the hook parameters) or we should deprecate this construct
as part of a bygone era.

I lean on deprecation because it feels like a thing we don't really want
to support going forward, but I can go either way.

-Sean

P.S. I'm starting to look at in tree functional testing for all of this,
in the event that we decide not to deprecate it. It's definitely made a
little hard by the way all exceptions are caught when hooks go wrong.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Sean Dague]

On 02/29/2016 05:23 PM, Matt Riedemann wrote:
> 
> 
> On 2/29/2016 2:54 PM, Sean Dague wrote:
>> On 02/29/2016 11:59 AM, Sean Dague wrote:
>>> The nova/hooks.py infrastructure has been with us since early Nova. It's
>>> currently only annotated on a few locations - 'build_instance',
>>> 'create_instance', 'delete_instance', and 'instance_network_info'. It's
>>> got a couple of unit tests on it, but nothing that actually tests real
>>> behavior of the hooks we have specified.
>>>
>>> It does get used in the wild, and we do break it with changes we didn't
>>> ever anticipate would impact it -
>>> https://bugs.launchpad.net/nova/+bug/1518321
>>>
>>> However, when you look into how that is used, it's really really odd and
>>> fragile -
>>> https://github.com/richm/rdo-vm-factory/blob/master/rdo-ipa-nova/novahooks.py#L248
>>>
>>>
>>>
>>>  def pre(self, *args, **kwargs):
>>>  # args[7] is the injected_files parameter array
>>>  # the value is ('filename', 'base64 encoded contents')
>>>  ipaotp = str(uuid.uuid4())
>>>  ipainject = ('/tmp/ipaotp', base64.b64encode(ipaotp))
>>>  args[7].extend(self.inject_files)
>>>  args[7].append(ipainject)
>>>
>>> In our continued quest on being more explicit about plug points it feels
>>> like we should other document the interface (which means creating
>>> stability on the hook parameters) or we should deprecate this construct
>>> as part of a bygone era.
>>>
>>> I lean on deprecation because it feels like a thing we don't really want
>>> to support going forward, but I can go either way.
>>>
>>> -Sean
>>>
>>> P.S. I'm starting to look at in tree functional testing for all of this,
>>> in the event that we decide not to deprecate it. It's definitely made a
>>> little hard by the way all exceptions are caught when hooks go wrong.
>>
>> As there seemed to be some early enthusiasm for this from the core team,
>> the deprecation patch is proposed here -
>> https://review.openstack.org/#/c/286276/
>>
>> Unless there is a big reversal of sentiment I'd suggest we get that
>> landed in Mitaka so that this is signalled to people going forward, and
>> we can collect use cases for things that are currently using hooks that
>> we may want to support in other ways in Newton.
>>
>> -Sean
>>
> 
> I think you should probably also post this to the operators mailing list
> before we actually deprecate hooks.

I'm not sure that's really true. As a team we have to decide we're going
to deprecate or document this facility. The status quo is really not a
thing.

So unless we've got folks that believe we should crystalize this
interface, and are signing up for doing that (which means you can't
change build_instance internal function parameters ever), then I think
it's fine to deprecate. We can measure our removal timeframe based on
reaction to that.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[John Garbutt]

On 29 February 2016 at 18:49, Andrew Laski  wrote:
> On Mon, Feb 29, 2016, at 01:18 PM, Dan Smith wrote:
>> > Forgive my ignorance or for playing devil's advocate, but wouldn't the
>> > main difference between notifications and hooks be that notifications
>> > are asynchronous and hooks aren't?
>>
>> The main difference is that notifications are external and intended to
>> be stable (especially with the versioned notifications effort). The
>> hooks are internal and depend wholly on internal data structures.
>>
>> > In the case of how Rdo was using it,
>> > they are adding things to the injected_files list before the instance is
>> > created in the compute API.  You couldn't do that with notifications as
>> > far as I know.
>>
>> Nope, definitely not, but I see that as a good thing. Injecting files
>> like that is likely to be very fragile and I think mostly regarded as
>> substantially less desirable than the alternatives, regardless of how it
>> happens.
>>
>> I think that Laski's point was that the most useful and least dangerous
>> thing that hooks can be used for is the use case that is much better
>> served by notifications.
>
> Yep. My experience with them was things like updating an external cache
> on create/delete or calling out to a DNS provider to remove a reverse
> DNS entry on instance delete. Things that could easily be handled with
> notifications, and use cases that I think we should continue to support
> by improving notifications if necessary.
>
>
>>
>> So, if file injection (and any other internals-mangling that other
>> people may use them for) is not a reason to keep hooks, and if
>> notifications are the proper way to trigger on things happening, then
>> there's no reason to keep hooks.

+1 on the deprecation of hooks.

There will not be a single replacement for the hooks.

Deprecation is the best thing we can do to trigger good conversations
about how we replacing hooks with a supportable alternatives, that
will not break horribly across upgrades.

This is an important part of our push towards improving API
interoperability, see:
http://docs.openstack.org/developer/nova/project_scope.html#api-scope

The OpenStack Compute API should behave the same across all
deployments (i.e. any differences should be discoverable).

Thanks,
johnthetubaguy

PS
Around DNS integrations, we have just added this:
https://blueprints.launchpad.net/nova/+spec/neutron-hostname-dns

Injecting files fights interoperability in a nasty way, but do take a
look at this:
https://github.com/openstack/nova/blob/master/nova/api/metadata/vendordata_json.py

The work around versioned notifications should make it possible to
build a async system from our notification stream:
http://specs.openstack.org/openstack/nova-specs/specs/mitaka/approved/versioned-notification-api.html

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[John Garbutt]

On 1 March 2016 at 10:10, Daniel P. Berrange  wrote:
> On Mon, Feb 29, 2016 at 12:36:03PM -0700, Rich Megginson wrote:
>> On 02/29/2016 12:19 PM, Chris Friesen wrote:
>> >On 02/29/2016 12:22 PM, Daniel P. Berrange wrote:
>> >
>> >>There's three core scenarios for hooks
>> >>
>> >>  1. Modifying some aspect of the Nova operation
>> >>  2. Triggering an external action synchronously to some Nova operation
>> >>  3. Triggering an external action asynchronously to some Nova operation
>> >>
>> >>The Rdo example is falling in scenario 1 since it is modifying the
>> >>injected files. I think this is is absolutely the kind of thing
>> >>we should explicitly *never* support. When external code can arbitrarily
>> >>modify some aspect of Nova operation we're in totally unchartered
>> >>territory as to the behaviour of Nova. To support that we'd need to
>> >>provide a stable internal API which is just not something we want to
>> >>tie ourselves into. I don't know just what the Rdo example is trying
>> >>to achieve, but whatever it is, it should be via some supportable API
>> >>and not a hook.,
>> >>
>> >>Scenaris 2 and 3 are both valid to consider. Using the notifications
>> >>system gets as an asynchronous trigger mechanism, which is probably
>> >>fine for many scenarios.  The big question is whether there's a
>> >>compelling need for scenario two, where the external action blocks
>> >>execution of the Nova operation until it has completed its hook.
>> >
>> >Even in the case of scenario two it is possible in some cases to use a
>> >proxy to intercept the HTTP request, take action, and then forward it or
>> >reject it as appropriate.
>> >
>> >I think the real question is whether there's a need to trigger an external
>> >action synchronously from down in the guts of the nova code.
>>
>> The hooks do the following: 
>> https://github.com/rcritten/rdo-vm-factory/blob/use-centos/rdo-ipa-nova/novahooks.py#L271
>>
>> We need to generate a token (ipaotp) and call ipa host-add with that token
>> _before_ the new machine has a chance to call ipa-client-install.  We have
>> to guarantee that the client cannot call ipa-client-install until we get
>> back the response from ipa that the host has been added with the token.
>> Additionally, we store the token in an injected_file in the new machine, so
>> the file can be removed as soon as possible.  We tried storing the token in
>> the VM metadata, but there is apparently no way to delete it?  Can the
>> machine do
>>
>> curl -XDELETE http://168.254.169.254/openstack/latest/metadata?key=value ?
>>
>> Using the build_instance.pre hook in Nova makes this simple and
>> straightforward.  It's also relatively painless to use the network_info.post
>> hook to handle the floating ip address assignment.
>>
>> Is it possible to do the above using notifications without jumping through
>> too many hoops?
>
> So from a high level POV, you are trying to generate a security token
> which will be provided to the guest OS before it is booted.
>
> I think that is a pretty clearly useful feature, and something that
> should really be officially integrated into Nova as a concept rather
> than done behind nova's back as a hook.

We did discuss the road to creating a very similar mechanism at the
last design summit, although the notes from that session are a little
cryptic:
https://etherpad.openstack.org/p/mitaka-nova-service-users

Roughly it was a per instance keystone token that would then give you
you access to Barbican, or similar.

Thanks,
johnthetubaguy

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Daniel P. Berrange]

On Mon, Feb 29, 2016 at 12:36:03PM -0700, Rich Megginson wrote:
> On 02/29/2016 12:19 PM, Chris Friesen wrote:
> >On 02/29/2016 12:22 PM, Daniel P. Berrange wrote:
> >
> >>There's three core scenarios for hooks
> >>
> >>  1. Modifying some aspect of the Nova operation
> >>  2. Triggering an external action synchronously to some Nova operation
> >>  3. Triggering an external action asynchronously to some Nova operation
> >>
> >>The Rdo example is falling in scenario 1 since it is modifying the
> >>injected files. I think this is is absolutely the kind of thing
> >>we should explicitly *never* support. When external code can arbitrarily
> >>modify some aspect of Nova operation we're in totally unchartered
> >>territory as to the behaviour of Nova. To support that we'd need to
> >>provide a stable internal API which is just not something we want to
> >>tie ourselves into. I don't know just what the Rdo example is trying
> >>to achieve, but whatever it is, it should be via some supportable API
> >>and not a hook.,
> >>
> >>Scenaris 2 and 3 are both valid to consider. Using the notifications
> >>system gets as an asynchronous trigger mechanism, which is probably
> >>fine for many scenarios.  The big question is whether there's a
> >>compelling need for scenario two, where the external action blocks
> >>execution of the Nova operation until it has completed its hook.
> >
> >Even in the case of scenario two it is possible in some cases to use a
> >proxy to intercept the HTTP request, take action, and then forward it or
> >reject it as appropriate.
> >
> >I think the real question is whether there's a need to trigger an external
> >action synchronously from down in the guts of the nova code.
> 
> The hooks do the following: 
> https://github.com/rcritten/rdo-vm-factory/blob/use-centos/rdo-ipa-nova/novahooks.py#L271
> 
> We need to generate a token (ipaotp) and call ipa host-add with that token
> _before_ the new machine has a chance to call ipa-client-install.  We have
> to guarantee that the client cannot call ipa-client-install until we get
> back the response from ipa that the host has been added with the token.
> Additionally, we store the token in an injected_file in the new machine, so
> the file can be removed as soon as possible.  We tried storing the token in
> the VM metadata, but there is apparently no way to delete it?  Can the
> machine do
> 
> curl -XDELETE http://168.254.169.254/openstack/latest/metadata?key=value ?
> 
> Using the build_instance.pre hook in Nova makes this simple and
> straightforward.  It's also relatively painless to use the network_info.post
> hook to handle the floating ip address assignment.
> 
> Is it possible to do the above using notifications without jumping through
> too many hoops?

So from a high level POV, you are trying to generate a security token
which will be provided to the guest OS before it is booted.

I think that is a pretty clearly useful feature, and something that
should really be officially integrated into Nova as a concept rather
than done behind nova's back as a hook.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Juan Antonio Osorio]

On Tue, Mar 1, 2016 at 12:23 AM, Matt Riedemann 
wrote:

>
>
> On 2/29/2016 2:54 PM, Sean Dague wrote:
>
>> On 02/29/2016 11:59 AM, Sean Dague wrote:
>>
>>> The nova/hooks.py infrastructure has been with us since early Nova. It's
>>> currently only annotated on a few locations - 'build_instance',
>>> 'create_instance', 'delete_instance', and 'instance_network_info'. It's
>>> got a couple of unit tests on it, but nothing that actually tests real
>>> behavior of the hooks we have specified.
>>>
>>> It does get used in the wild, and we do break it with changes we didn't
>>> ever anticipate would impact it -
>>> https://bugs.launchpad.net/nova/+bug/1518321
>>>
>>> However, when you look into how that is used, it's really really odd and
>>> fragile -
>>>
>>> https://github.com/richm/rdo-vm-factory/blob/master/rdo-ipa-nova/novahooks.py#L248
>>>
>>>
>>>  def pre(self, *args, **kwargs):
>>>  # args[7] is the injected_files parameter array
>>>  # the value is ('filename', 'base64 encoded contents')
>>>  ipaotp = str(uuid.uuid4())
>>>  ipainject = ('/tmp/ipaotp', base64.b64encode(ipaotp))
>>>  args[7].extend(self.inject_files)
>>>  args[7].append(ipainject)
>>>
>>> In our continued quest on being more explicit about plug points it feels
>>> like we should other document the interface (which means creating
>>> stability on the hook parameters) or we should deprecate this construct
>>> as part of a bygone era.
>>>
>>> I lean on deprecation because it feels like a thing we don't really want
>>> to support going forward, but I can go either way.
>>>
>>> -Sean
>>>
>>> P.S. I'm starting to look at in tree functional testing for all of this,
>>> in the event that we decide not to deprecate it. It's definitely made a
>>> little hard by the way all exceptions are caught when hooks go wrong.
>>>
>>
>> As there seemed to be some early enthusiasm for this from the core team,
>> the deprecation patch is proposed here -
>> https://review.openstack.org/#/c/286276/
>>
>> Unless there is a big reversal of sentiment I'd suggest we get that
>> landed in Mitaka so that this is signalled to people going forward, and
>> we can collect use cases for things that are currently using hooks that
>> we may want to support in other ways in Newton.
>>
>> -Sean
>>
>>
> I think you should probably also post this to the operators mailing list
> before we actually deprecate hooks.


I agree with Matt, I think this should get more visibility with the
operators first before deprecating it.


>
> --
>
> Thanks,
>
> Matt Riedemann
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Juan Antonio Osorio R.
e-mail: jaosor...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Matt Riedemann]

On 2/29/2016 2:54 PM, Sean Dague wrote:

On 02/29/2016 11:59 AM, Sean Dague wrote:

The nova/hooks.py infrastructure has been with us since early Nova. It's
currently only annotated on a few locations - 'build_instance',
'create_instance', 'delete_instance', and 'instance_network_info'. It's
got a couple of unit tests on it, but nothing that actually tests real
behavior of the hooks we have specified.

It does get used in the wild, and we do break it with changes we didn't
ever anticipate would impact it -
https://bugs.launchpad.net/nova/+bug/1518321

However, when you look into how that is used, it's really really odd and
fragile -
https://github.com/richm/rdo-vm-factory/blob/master/rdo-ipa-nova/novahooks.py#L248


 def pre(self, *args, **kwargs):
 # args[7] is the injected_files parameter array
 # the value is ('filename', 'base64 encoded contents')
 ipaotp = str(uuid.uuid4())
 ipainject = ('/tmp/ipaotp', base64.b64encode(ipaotp))
 args[7].extend(self.inject_files)
 args[7].append(ipainject)

In our continued quest on being more explicit about plug points it feels
like we should other document the interface (which means creating
stability on the hook parameters) or we should deprecate this construct
as part of a bygone era.

I lean on deprecation because it feels like a thing we don't really want
to support going forward, but I can go either way.

-Sean

P.S. I'm starting to look at in tree functional testing for all of this,
in the event that we decide not to deprecate it. It's definitely made a
little hard by the way all exceptions are caught when hooks go wrong.


As there seemed to be some early enthusiasm for this from the core team,
the deprecation patch is proposed here -
https://review.openstack.org/#/c/286276/

Unless there is a big reversal of sentiment I'd suggest we get that
landed in Mitaka so that this is signalled to people going forward, and
we can collect use cases for things that are currently using hooks that
we may want to support in other ways in Newton.

-Sean



I think you should probably also post this to the operators mailing list 
before we actually deprecate hooks.


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Sean Dague]

On 02/29/2016 11:59 AM, Sean Dague wrote:
> The nova/hooks.py infrastructure has been with us since early Nova. It's
> currently only annotated on a few locations - 'build_instance',
> 'create_instance', 'delete_instance', and 'instance_network_info'. It's
> got a couple of unit tests on it, but nothing that actually tests real
> behavior of the hooks we have specified.
> 
> It does get used in the wild, and we do break it with changes we didn't
> ever anticipate would impact it -
> https://bugs.launchpad.net/nova/+bug/1518321
> 
> However, when you look into how that is used, it's really really odd and
> fragile -
> https://github.com/richm/rdo-vm-factory/blob/master/rdo-ipa-nova/novahooks.py#L248
> 
> 
> def pre(self, *args, **kwargs):
> # args[7] is the injected_files parameter array
> # the value is ('filename', 'base64 encoded contents')
> ipaotp = str(uuid.uuid4())
> ipainject = ('/tmp/ipaotp', base64.b64encode(ipaotp))
> args[7].extend(self.inject_files)
> args[7].append(ipainject)
> 
> In our continued quest on being more explicit about plug points it feels
> like we should other document the interface (which means creating
> stability on the hook parameters) or we should deprecate this construct
> as part of a bygone era.
> 
> I lean on deprecation because it feels like a thing we don't really want
> to support going forward, but I can go either way.
> 
>   -Sean
> 
> P.S. I'm starting to look at in tree functional testing for all of this,
> in the event that we decide not to deprecate it. It's definitely made a
> little hard by the way all exceptions are caught when hooks go wrong.

As there seemed to be some early enthusiasm for this from the core team,
the deprecation patch is proposed here -
https://review.openstack.org/#/c/286276/

Unless there is a big reversal of sentiment I'd suggest we get that
landed in Mitaka so that this is signalled to people going forward, and
we can collect use cases for things that are currently using hooks that
we may want to support in other ways in Newton.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Rich Megginson]

On 02/29/2016 12:19 PM, Chris Friesen wrote:

On 02/29/2016 12:22 PM, Daniel P. Berrange wrote:


There's three core scenarios for hooks

  1. Modifying some aspect of the Nova operation
  2. Triggering an external action synchronously to some Nova operation
  3. Triggering an external action asynchronously to some Nova operation

The Rdo example is falling in scenario 1 since it is modifying the
injected files. I think this is is absolutely the kind of thing
we should explicitly *never* support. When external code can arbitrarily
modify some aspect of Nova operation we're in totally unchartered
territory as to the behaviour of Nova. To support that we'd need to
provide a stable internal API which is just not something we want to
tie ourselves into. I don't know just what the Rdo example is trying
to achieve, but whatever it is, it should be via some supportable API
and not a hook.,

Scenaris 2 and 3 are both valid to consider. Using the notifications
system gets as an asynchronous trigger mechanism, which is probably
fine for many scenarios.  The big question is whether there's a
compelling need for scenario two, where the external action blocks
execution of the Nova operation until it has completed its hook.


Even in the case of scenario two it is possible in some cases to use a 
proxy to intercept the HTTP request, take action, and then forward it 
or reject it as appropriate.


I think the real question is whether there's a need to trigger an 
external action synchronously from down in the guts of the nova code.


The hooks do the following: 
https://github.com/rcritten/rdo-vm-factory/blob/use-centos/rdo-ipa-nova/novahooks.py#L271


We need to generate a token (ipaotp) and call ipa host-add with that 
token _before_ the new machine has a chance to call ipa-client-install.  
We have to guarantee that the client cannot call ipa-client-install 
until we get back the response from ipa that the host has been added 
with the token.  Additionally, we store the token in an injected_file in 
the new machine, so the file can be removed as soon as possible.  We 
tried storing the token in the VM metadata, but there is apparently no 
way to delete it?  Can the machine do


curl -XDELETE http://168.254.169.254/openstack/latest/metadata?key=value ?

Using the build_instance.pre hook in Nova makes this simple and 
straightforward.  It's also relatively painless to use the 
network_info.post hook to handle the floating ip address assignment.


Is it possible to do the above using notifications without jumping 
through too many hoops?




Chris

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Chris Friesen]

On 02/29/2016 12:22 PM, Daniel P. Berrange wrote:


There's three core scenarios for hooks

  1. Modifying some aspect of the Nova operation
  2. Triggering an external action synchronously to some Nova operation
  3. Triggering an external action asynchronously to some Nova operation

The Rdo example is falling in scenario 1 since it is modifying the
injected files. I think this is is absolutely the kind of thing
we should explicitly *never* support. When external code can arbitrarily
modify some aspect of Nova operation we're in totally unchartered
territory as to the behaviour of Nova. To support that we'd need to
provide a stable internal API which is just not something we want to
tie ourselves into. I don't know just what the Rdo example is trying
to achieve, but whatever it is, it should be via some supportable API
and not a hook.,

Scenaris 2 and 3 are both valid to consider. Using the notifications
system gets as an asynchronous trigger mechanism, which is probably
fine for many scenarios.  The big question is whether there's a
compelling need for scenario two, where the external action blocks
execution of the Nova operation until it has completed its hook.


Even in the case of scenario two it is possible in some cases to use a proxy to 
intercept the HTTP request, take action, and then forward it or reject it as 
appropriate.


I think the real question is whether there's a need to trigger an external 
action synchronously from down in the guts of the nova code.


Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Andrew Laski]

On Mon, Feb 29, 2016, at 01:18 PM, Dan Smith wrote:
> > Forgive my ignorance or for playing devil's advocate, but wouldn't the
> > main difference between notifications and hooks be that notifications
> > are asynchronous and hooks aren't?
> 
> The main difference is that notifications are external and intended to
> be stable (especially with the versioned notifications effort). The
> hooks are internal and depend wholly on internal data structures.
> 
> > In the case of how Rdo was using it,
> > they are adding things to the injected_files list before the instance is
> > created in the compute API.  You couldn't do that with notifications as
> > far as I know.
> 
> Nope, definitely not, but I see that as a good thing. Injecting files
> like that is likely to be very fragile and I think mostly regarded as
> substantially less desirable than the alternatives, regardless of how it
> happens.
> 
> I think that Laski's point was that the most useful and least dangerous
> thing that hooks can be used for is the use case that is much better
> served by notifications.

Yep. My experience with them was things like updating an external cache
on create/delete or calling out to a DNS provider to remove a reverse
DNS entry on instance delete. Things that could easily be handled with
notifications, and use cases that I think we should continue to support
by improving notifications if necessary.


> 
> So, if file injection (and any other internals-mangling that other
> people may use them for) is not a reason to keep hooks, and if
> notifications are the proper way to trigger on things happening, then
> there's no reason to keep hooks.
> 
> --Dan
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Sean Dague]

On 02/29/2016 01:33 PM, Andrew Laski wrote:
> 
> 
> On Mon, Feb 29, 2016, at 12:53 PM, Rob Crittenden wrote:
>> Andrew Laski wrote:
>>>
>>>
>>> On Mon, Feb 29, 2016, at 12:12 PM, Dan Smith wrote:
> In our continued quest on being more explicit about plug points it feels
> like we should other document the interface (which means creating
> stability on the hook parameters) or we should deprecate this construct
> as part of a bygone era.
>
> I lean on deprecation because it feels like a thing we don't really want
> to support going forward, but I can go either way.

 Deprecate and remove, please. We've been removing these sorts of things
 over time, and nova hooks have been ignored in that process. But really,
 making them more rigid is going to get in the way over time, trying to
 continue to honor an interface that codifies internals at a certain
 point in time, and leaving them as-is will just continue to generate
 issues like the quoted bug.

 I don't "lean" on deprecation, I feel strongly that these should go away.
>>>
>>> I've worked on a deployment that uses them heavily and would be impacted
>>> by their removal. They are a very convenient place to put code that
>>> should run based on Nova events but I have yet to see a usage that
>>> couldn't have been implemented by having a service listen to
>>> notifications and run that same code. However there is no service that
>>> does this. So the only argument I can see for keeping them is that it's
>>> more convenient to put that code into Nova rather than implement
>>> something that listens for notifications. And that's not a convincing
>>> argument to me.
>>
>> The ability to inject files on a per-VM basis is one thing that would be
>> missing. Right now decisions can be made inside the hook using
>> attributes of the VM to decide whether and what to inject, including
>> files generated inside the hook itself.
>>
>> I won't argue that the API is nice, it isn't, but I'd prefer a replace
>> then deprecate model instead.
>>
>> I'm not entirely sure that the notifications contain all the same
>> information as is available now in the hooks.
> 
> This is something we can and should address if there is useful
> information missing from notifications.

100% agreed. I would consider that a priority feature for Newton if it
can be enumerated.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Andrew Laski]

On Mon, Feb 29, 2016, at 12:53 PM, Rob Crittenden wrote:
> Andrew Laski wrote:
> > 
> > 
> > On Mon, Feb 29, 2016, at 12:12 PM, Dan Smith wrote:
> >>> In our continued quest on being more explicit about plug points it feels
> >>> like we should other document the interface (which means creating
> >>> stability on the hook parameters) or we should deprecate this construct
> >>> as part of a bygone era.
> >>>
> >>> I lean on deprecation because it feels like a thing we don't really want
> >>> to support going forward, but I can go either way.
> >>
> >> Deprecate and remove, please. We've been removing these sorts of things
> >> over time, and nova hooks have been ignored in that process. But really,
> >> making them more rigid is going to get in the way over time, trying to
> >> continue to honor an interface that codifies internals at a certain
> >> point in time, and leaving them as-is will just continue to generate
> >> issues like the quoted bug.
> >>
> >> I don't "lean" on deprecation, I feel strongly that these should go away.
> > 
> > I've worked on a deployment that uses them heavily and would be impacted
> > by their removal. They are a very convenient place to put code that
> > should run based on Nova events but I have yet to see a usage that
> > couldn't have been implemented by having a service listen to
> > notifications and run that same code. However there is no service that
> > does this. So the only argument I can see for keeping them is that it's
> > more convenient to put that code into Nova rather than implement
> > something that listens for notifications. And that's not a convincing
> > argument to me.
> 
> The ability to inject files on a per-VM basis is one thing that would be
> missing. Right now decisions can be made inside the hook using
> attributes of the VM to decide whether and what to inject, including
> files generated inside the hook itself.
> 
> I won't argue that the API is nice, it isn't, but I'd prefer a replace
> then deprecate model instead.
> 
> I'm not entirely sure that the notifications contain all the same
> information as is available now in the hooks.

This is something we can and should address if there is useful
information missing from notifications.

> 
> rob
> 
> > So I agree with moving forward on deprecation and think that
> > notifications provide a suitable replacement for the functionality
> > provided.
> > 
> > 
> >>
> >> --Dan
> >>
> >> __
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Daniel P. Berrange]

On Mon, Feb 29, 2016 at 12:03:00PM -0600, Matt Riedemann wrote:
> 
> 
> On 2/29/2016 11:23 AM, Andrew Laski wrote:
> >
> >
> >On Mon, Feb 29, 2016, at 12:12 PM, Dan Smith wrote:
> >>>In our continued quest on being more explicit about plug points it feels
> >>>like we should other document the interface (which means creating
> >>>stability on the hook parameters) or we should deprecate this construct
> >>>as part of a bygone era.
> >>>
> >>>I lean on deprecation because it feels like a thing we don't really want
> >>>to support going forward, but I can go either way.
> >>
> >>Deprecate and remove, please. We've been removing these sorts of things
> >>over time, and nova hooks have been ignored in that process. But really,
> >>making them more rigid is going to get in the way over time, trying to
> >>continue to honor an interface that codifies internals at a certain
> >>point in time, and leaving them as-is will just continue to generate
> >>issues like the quoted bug.
> >>
> >>I don't "lean" on deprecation, I feel strongly that these should go away.
> >
> >I've worked on a deployment that uses them heavily and would be impacted
> >by their removal. They are a very convenient place to put code that
> >should run based on Nova events but I have yet to see a usage that
> >couldn't have been implemented by having a service listen to
> >notifications and run that same code. However there is no service that
> >does this. So the only argument I can see for keeping them is that it's
> >more convenient to put that code into Nova rather than implement
> >something that listens for notifications. And that's not a convincing
> >argument to me.
> >
> >So I agree with moving forward on deprecation and think that
> >notifications provide a suitable replacement for the functionality
> >provided.
> 
> Forgive my ignorance or for playing devil's advocate, but wouldn't the main
> difference between notifications and hooks be that notifications are
> asynchronous and hooks aren't?  In the case of how Rdo was using it, they
> are adding things to the injected_files list before the instance is created
> in the compute API.  You couldn't do that with notifications as far as I
> know.

Sure, notifications do *not* replace all possible use case of the current
hooks, but I think that is actually desirable.

There's three core scenarios for hooks

 1. Modifying some aspect of the Nova operation
 2. Triggering an external action synchronously to some Nova operation
 3. Triggering an external action asynchronously to some Nova operation

The Rdo example is falling in scenario 1 since it is modifying the
injected files. I think this is is absolutely the kind of thing
we should explicitly *never* support. When external code can arbitrarily
modify some aspect of Nova operation we're in totally unchartered
territory as to the behaviour of Nova. To support that we'd need to
provide a stable internal API which is just not something we want to
tie ourselves into. I don't know just what the Rdo example is trying
to achieve, but whatever it is, it should be via some supportable API
and not a hook.,

Scenaris 2 and 3 are both valid to consider. Using the notifications
system gets as an asynchronous trigger mechanism, which is probably
fine for many scenarios.  The big question is whether there's a
compelling need for scenario two, where the external action blocks
execution of the Nova operation until it has completed its hook.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Dan Smith]

> Forgive my ignorance or for playing devil's advocate, but wouldn't the
> main difference between notifications and hooks be that notifications
> are asynchronous and hooks aren't?

The main difference is that notifications are external and intended to
be stable (especially with the versioned notifications effort). The
hooks are internal and depend wholly on internal data structures.

> In the case of how Rdo was using it,
> they are adding things to the injected_files list before the instance is
> created in the compute API.  You couldn't do that with notifications as
> far as I know.

Nope, definitely not, but I see that as a good thing. Injecting files
like that is likely to be very fragile and I think mostly regarded as
substantially less desirable than the alternatives, regardless of how it
happens.

I think that Laski's point was that the most useful and least dangerous
thing that hooks can be used for is the use case that is much better
served by notifications.

So, if file injection (and any other internals-mangling that other
people may use them for) is not a reason to keep hooks, and if
notifications are the proper way to trigger on things happening, then
there's no reason to keep hooks.

--Dan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Matt Riedemann]

On 2/29/2016 11:23 AM, Andrew Laski wrote:



On Mon, Feb 29, 2016, at 12:12 PM, Dan Smith wrote:

In our continued quest on being more explicit about plug points it feels
like we should other document the interface (which means creating
stability on the hook parameters) or we should deprecate this construct
as part of a bygone era.

I lean on deprecation because it feels like a thing we don't really want
to support going forward, but I can go either way.


Deprecate and remove, please. We've been removing these sorts of things
over time, and nova hooks have been ignored in that process. But really,
making them more rigid is going to get in the way over time, trying to
continue to honor an interface that codifies internals at a certain
point in time, and leaving them as-is will just continue to generate
issues like the quoted bug.

I don't "lean" on deprecation, I feel strongly that these should go away.


I've worked on a deployment that uses them heavily and would be impacted
by their removal. They are a very convenient place to put code that
should run based on Nova events but I have yet to see a usage that
couldn't have been implemented by having a service listen to
notifications and run that same code. However there is no service that
does this. So the only argument I can see for keeping them is that it's
more convenient to put that code into Nova rather than implement
something that listens for notifications. And that's not a convincing
argument to me.

So I agree with moving forward on deprecation and think that
notifications provide a suitable replacement for the functionality
provided.




--Dan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



Forgive my ignorance or for playing devil's advocate, but wouldn't the 
main difference between notifications and hooks be that notifications 
are asynchronous and hooks aren't?  In the case of how Rdo was using it, 
they are adding things to the injected_files list before the instance is 
created in the compute API.  You couldn't do that with notifications as 
far as I know.


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Rob Crittenden]

Andrew Laski wrote:
> 
> 
> On Mon, Feb 29, 2016, at 12:12 PM, Dan Smith wrote:
>>> In our continued quest on being more explicit about plug points it feels
>>> like we should other document the interface (which means creating
>>> stability on the hook parameters) or we should deprecate this construct
>>> as part of a bygone era.
>>>
>>> I lean on deprecation because it feels like a thing we don't really want
>>> to support going forward, but I can go either way.
>>
>> Deprecate and remove, please. We've been removing these sorts of things
>> over time, and nova hooks have been ignored in that process. But really,
>> making them more rigid is going to get in the way over time, trying to
>> continue to honor an interface that codifies internals at a certain
>> point in time, and leaving them as-is will just continue to generate
>> issues like the quoted bug.
>>
>> I don't "lean" on deprecation, I feel strongly that these should go away.
> 
> I've worked on a deployment that uses them heavily and would be impacted
> by their removal. They are a very convenient place to put code that
> should run based on Nova events but I have yet to see a usage that
> couldn't have been implemented by having a service listen to
> notifications and run that same code. However there is no service that
> does this. So the only argument I can see for keeping them is that it's
> more convenient to put that code into Nova rather than implement
> something that listens for notifications. And that's not a convincing
> argument to me.

The ability to inject files on a per-VM basis is one thing that would be
missing. Right now decisions can be made inside the hook using
attributes of the VM to decide whether and what to inject, including
files generated inside the hook itself.

I won't argue that the API is nice, it isn't, but I'd prefer a replace
then deprecate model instead.

I'm not entirely sure that the notifications contain all the same
information as is available now in the hooks.

rob

> So I agree with moving forward on deprecation and think that
> notifications provide a suitable replacement for the functionality
> provided.
> 
> 
>>
>> --Dan
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Daniel P. Berrange]

On Mon, Feb 29, 2016 at 12:23:09PM -0500, Andrew Laski wrote:
> 
> 
> On Mon, Feb 29, 2016, at 12:12 PM, Dan Smith wrote:
> > > In our continued quest on being more explicit about plug points it feels
> > > like we should other document the interface (which means creating
> > > stability on the hook parameters) or we should deprecate this construct
> > > as part of a bygone era.
> > > 
> > > I lean on deprecation because it feels like a thing we don't really want
> > > to support going forward, but I can go either way.
> > 
> > Deprecate and remove, please. We've been removing these sorts of things
> > over time, and nova hooks have been ignored in that process. But really,
> > making them more rigid is going to get in the way over time, trying to
> > continue to honor an interface that codifies internals at a certain
> > point in time, and leaving them as-is will just continue to generate
> > issues like the quoted bug.
> > 
> > I don't "lean" on deprecation, I feel strongly that these should go away.
> 
> I've worked on a deployment that uses them heavily and would be impacted
> by their removal. They are a very convenient place to put code that
> should run based on Nova events but I have yet to see a usage that
> couldn't have been implemented by having a service listen to
> notifications and run that same code. However there is no service that
> does this. So the only argument I can see for keeping them is that it's
> more convenient to put that code into Nova rather than implement
> something that listens for notifications. And that's not a convincing
> argument to me.

Yes, that's a prime example of a use case where we should be offering
a formally supportable solution instead of an unreliable hack using
hooks.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Andrew Laski]

On Mon, Feb 29, 2016, at 12:12 PM, Dan Smith wrote:
> > In our continued quest on being more explicit about plug points it feels
> > like we should other document the interface (which means creating
> > stability on the hook parameters) or we should deprecate this construct
> > as part of a bygone era.
> > 
> > I lean on deprecation because it feels like a thing we don't really want
> > to support going forward, but I can go either way.
> 
> Deprecate and remove, please. We've been removing these sorts of things
> over time, and nova hooks have been ignored in that process. But really,
> making them more rigid is going to get in the way over time, trying to
> continue to honor an interface that codifies internals at a certain
> point in time, and leaving them as-is will just continue to generate
> issues like the quoted bug.
> 
> I don't "lean" on deprecation, I feel strongly that these should go away.

I've worked on a deployment that uses them heavily and would be impacted
by their removal. They are a very convenient place to put code that
should run based on Nova events but I have yet to see a usage that
couldn't have been implemented by having a service listen to
notifications and run that same code. However there is no service that
does this. So the only argument I can see for keeping them is that it's
more convenient to put that code into Nova rather than implement
something that listens for notifications. And that's not a convincing
argument to me.

So I agree with moving forward on deprecation and think that
notifications provide a suitable replacement for the functionality
provided.


> 
> --Dan
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Daniel P. Berrange]

On Mon, Feb 29, 2016 at 11:59:06AM -0500, Sean Dague wrote:
> The nova/hooks.py infrastructure has been with us since early Nova. It's
> currently only annotated on a few locations - 'build_instance',
> 'create_instance', 'delete_instance', and 'instance_network_info'. It's
> got a couple of unit tests on it, but nothing that actually tests real
> behavior of the hooks we have specified.
> 
> It does get used in the wild, and we do break it with changes we didn't
> ever anticipate would impact it -
> https://bugs.launchpad.net/nova/+bug/1518321
> 
> However, when you look into how that is used, it's really really odd and
> fragile -
> https://github.com/richm/rdo-vm-factory/blob/master/rdo-ipa-nova/novahooks.py#L248
> 
> 
> def pre(self, *args, **kwargs):
> # args[7] is the injected_files parameter array
> # the value is ('filename', 'base64 encoded contents')
> ipaotp = str(uuid.uuid4())
> ipainject = ('/tmp/ipaotp', base64.b64encode(ipaotp))
> args[7].extend(self.inject_files)
> args[7].append(ipainject)
> 
> In our continued quest on being more explicit about plug points it feels
> like we should other document the interface (which means creating
> stability on the hook parameters) or we should deprecate this construct
> as part of a bygone era.
> 
> I lean on deprecation because it feels like a thing we don't really want
> to support going forward, but I can go either way.

As it is designed, I think the hooks mechanism is really unsupportable
long term. It is exposing users to arbitrary internal Nova data structures
which we have changed at will and we cannot possibly ever consider them
to be a stable consumable API. I'm rather surprised we've not seen more
bugs like the one you've shown above - most likely thats a reflection
on not many people actually using this facility.

I'd be strongly in favour of deprecation & removal of this hooking
mechanism, as its unsupportable in any sane manner when it exposes
code to our internal unstable API & object model.

If there's stuff people are doing in hooks that's impossible any other
way, we should really be looking at what we need to provide in our
public API to achieve the same goal, if it is use case we wish to be
able to support.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Juan Antonio Osorio]

Well,

I do feel that the hooks have been pretty painful (having tried to use them
and
having stumbled upon that bug). However, even as undocumented as they were,
that seems to be the only way of doing actions in nova when instances are
spawned or deleted, while also having some awareness of what's given to the
instance. So we would need a replacement.

BR

On Mon, Feb 29, 2016 at 7:12 PM, Dan Smith  wrote:

> > In our continued quest on being more explicit about plug points it feels
> > like we should other document the interface (which means creating
> > stability on the hook parameters) or we should deprecate this construct
> > as part of a bygone era.
> >
> > I lean on deprecation because it feels like a thing we don't really want
> > to support going forward, but I can go either way.
>
> Deprecate and remove, please. We've been removing these sorts of things
> over time, and nova hooks have been ignored in that process. But really,
> making them more rigid is going to get in the way over time, trying to
> continue to honor an interface that codifies internals at a certain
> point in time, and leaving them as-is will just continue to generate
> issues like the quoted bug.
>
> I don't "lean" on deprecation, I feel strongly that these should go away.
>
> --Dan
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Juan Antonio Osorio R.
e-mail: jaosor...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

[Dan Smith]

> In our continued quest on being more explicit about plug points it feels
> like we should other document the interface (which means creating
> stability on the hook parameters) or we should deprecate this construct
> as part of a bygone era.
> 
> I lean on deprecation because it feels like a thing we don't really want
> to support going forward, but I can go either way.

Deprecate and remove, please. We've been removing these sorts of things
over time, and nova hooks have been ignored in that process. But really,
making them more rigid is going to get in the way over time, trying to
continue to honor an interface that codifies internals at a certain
point in time, and leaving them as-is will just continue to generate
issues like the quoted bug.

I don't "lean" on deprecation, I feel strongly that these should go away.

--Dan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev