Re: [openstack-dev] [Neutron] Neutron extenstions

[Christopher Yeoh]

Hi Doug,
On Thu, 19 Mar 2015 10:01:54 -0600
Doug Wiegley  wrote:

> Hi Gary,
> 
> First I’m seeing these, but I don’t see that they’re required on
> input, unless I’m mis-reading those reviews.  Additional of new
> output fields to a json object, or adding optional inputs, is not
> generally considered to be backwards incompatible behavior in an API.
> Does OpenStack have a stricter standard on that?

So speaking from a Nova (and more general) API point of view where
we're probably a bit longer along the extensions/microversioning path,
I think even with backwards compatibility it is still an issue. For
example, without some sort of co-operative and strong input validation
are you going to detect when someone sends a new parameter that is
simply mispelled and all the other extensions assume that someone will
look after it and return an appropriate rather than let it just through.

Over time we've found a few examples even in our api request samples
typos (which feeds into the docs) which are not detected just because
of this situation.

Chris

> 
> Thanks,
> doug
> 
> 
> > On Mar 19, 2015, at 6:37 AM, Gary Kotton  wrote:
> > 
> > Hi,
> > Changed the subject so that it may draw a little attention.
> > There were 2 patches approved that kind of break the API (in my
> > humble opinion): https://review.openstack.org/#/c/154921
> > / and
> > https://review.openstack.org/#/c/158420
> > / In both of these two new
> > fields were added to the base attributes – mtu and
> > vlan_transparency Reverts for them are:
> > https://review.openstack.org/165801
> >  (mtu) and
> > https://review.openstack.org/165776
> >  (vlan transparency). In my
> > opinion these should be added as separate extensions. Thanks Gary
> > 
> > From: Gary Kotton mailto:gkot...@vmware.com>>
> > Reply-To: OpenStack List  > > Date: Thursday, March
> > 19, 2015 at 2:32 PM To: OpenStack List
> >  > > Subject: Re:
> > [openstack-dev] [Neutron] VLAN transparency support
> > 
> > Hi,
> > This patch has the same addition too -
> > https://review.openstack.org/#/c/154921
> > /. We should also revert
> > that one. Thanks Gary
> > 
> > From: Gary Kotton mailto:gkot...@vmware.com>>
> > Reply-To: OpenStack List  > > Date: Thursday, March
> > 19, 2015 at 1:14 PM To: OpenStack List
> >  > > Subject:
> > [openstack-dev] [Neutron] VLAN transparency support
> > 
> > Hi,
> > It appears that https://review.openstack.org/#/c/158420
> > / update the base
> > attributes for the networks. Is there any reason why this was not
> > added as a separate extension like all others. I do not think that
> > this is the correct way to go and we should do this as all other
> > extensions have been maintained. I have posted a revert
> > (https://review.openstack.org/#/c/165776
> > /) – please feel free to
> > knack if it is invalid. Thanks Gary
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Christopher Yeoh]

On Tue, Mar 24, 2015 at 5:45 AM, Jay Pipes  wrote:

> On Tue, Mar 10, 2015 at 09:52:14PM +1030, Christopher Yeoh wrote:
> > On Mon, 09 Mar 2015 16:14:21 -0400
> > Sean Dague  wrote:
> >
> > > On 03/09/2015 03:37 PM, Jay Pipes wrote:
> > > > On 03/08/2015 08:10 AM, Alex Xu wrote:
> > > >> Thanks for Jay point this out! If we have agreement on this and
> > > >> document it, that will be great for guiding developer how to add
> > > >> new API.
> > > >>
> > > >> I know we didn't want extension for API. But I think we still
> > > >> need modularity. I don't think we should put everything in a single
> > > >> file, that file will become huge in the future and hard to
> > > >> maintenance.
> > > >
> > > > I don't think everything should be in a single file either. In fact,
> > > > I've never advocated for that.
> > > >
> > > >> We can make the 'extension' not configurable. Replace 'extension'
> > > >> with another name, deprecate the extension info api int he
> > > >> future But that is not mean we should put everything in a file.
> > > >
> > > > I didn't say that in my email. I'm not sure where you got the
> > > > impression I want to put everything in one file?
> > > >
> > > >> For modularity, we need define what should be in a separated
> > > >> module(it is extension now.) There are three cases:
> > > >>
> > > >> 1. Add new resource
> > > >>  This is totally worth to put in a separated module.
> > > >
> > > > Agreed.
> > > >
> > > >> 2. Add new sub-resource
> > > >>  like server-tags, I prefer to put in a separated module, I
> > > >> don't think put another 100 lines code in the servers.py is good
> > > >> choice.
> > > >
> > > > Agreed, which is exactly what I said in my email:
> > > >
> > > > "Similarly, new microversion API functionality should live in a
> > > > module, as a top-level (or subcollection) Controller in
> > > > /nova/api/openstack/compute/, and should not be in the
> > > > /nova/api/openstack/compute/plugins/ directory. Why? Because it's
> > > > not a plugin."
> > > >
> >
> > Ok so I'm getting confused about what we're disagreeing about then.
>
> I wasn't disagreeing with you. I was disagreeing with Alex Xu :) Perhaps
> your mail client was confusing you?
>
> > However, the first microversion change
> > https://review.openstack.org/#/c/140313/32 is one case where we didn't
> > need to create a new extension, relying only on microversions to add a
> > new parameter to the response, whereas server tags does add a new
> > conroller (os-server-tags) which is non trivia so I think it does. Do
> > we disagree about that?
>
> The server tags patch does add more functionality to the API, for sure.
> But, as I've written about in this thread, there's no reason at all that
> it needed to use the extension framework. It could better have been
> written as a simple Controller object in a new file in
> /nova/api/openstack/compute/server_tags.py, with no use of the
> extension mechanism whatsoever.
>
> > btw in a situation now where (I think) we are saying we are not going
> > to do any work for 3rd parties to add their own API plugins and have a
> > well planned API we don't need the prefixes as we don't need the prefix
> > on parameter names as there won't be name clashes without us realising
> > during testing. And we certainly don't need the os- prefix is the
> > plugin alias, but we do neeed them unique across the API I believe
> > because of the way we store information about them.
>
> Right. There won't be any more name clashes if we don't allow API
> extensions any more. Which is what I'm asking for.
>
> > > >> 3. extend attributes and methods for a existed resource
> > > >> like add new attributes for servers, we can choice one of
> > > >> existed module to put it in. Just like this patch
> > > >> https://review.openstack.org/#/c/155853/
> > > >> But for servers-tags, it's sub-resource, we can put it in
> > > >> its-own module.
> > > >
> > > > Agreed, and that's what I put in my email.
> > > >
> > &

Re: [openstack-dev] [nova] what is our shipped paste.ini going to be for Kilo

[Christopher Yeoh]

On Mon, 16 Mar 2015 10:15:50 -0400
Sean Dague  wrote:

> Our current top level shipped example paste.ini for Nova includes the
> following set of endpoint definitions:
> 
> [composite:osapi_compute]
> use = call:nova.api.openstack.urlmap:urlmap_factory
> /: oscomputeversions
> /v1.1: openstack_compute_api_v2
> /v2: openstack_compute_api_v2
> /v2.1: openstack_compute_api_v21
> /v3: openstack_compute_api_v3
> 
> 
> The real question I have is what should this look like in the Kilo
> release. And this has a couple of axes.
> 
>  Who uses our paste.ini?
> =
> 
> paste.ini is an etc file, so we assume that during upgrade you'll be
> using your existing config. Changes to the default paste.ini will
> really only be effective in new deploys. So this should not impact
> existing users, but instead only new users.
> 
> Cleaning up Cruft
> =
> 
> Drop of /v3
> ---
> 
> v3 is no longer a supported thing. I think v3 in the paste.ini causes
> confusion. It also causes us to keep around untested / unsupported
> code.
> 
> This seems kind of a no brainer.
> 
> Drop of /v1.1 ?
> ---
> 
> Only new deploys are really going to base off of our in tree
> config. I'm not convinced that we want to encourage people setting up
> new /v1.1 endpoint in tree.

+1 to dropping this. Can we also drop novaclient support for v1.1 at
some point. I too would like to know if anyone actually uses it


> 
> I'm not convinced there is a deprecation path here because the ones I
> could imagine would involve people changing their paste.ini to include
> a deprecation checking piece of code.
> 
> Honestly, I don't care strongly either way on this one. It's cruft,
> but not dangerous cruft (unlike v3).
> 
> Nova v2
> ===
> 
> This is where things get interesting.
> 
> v2.1 is supposed to be equivalent to v2. The difference is moving the
> validation for datastructures from the database to the wsgi layer. The
> ways in which this doesn't react like the existing APIs should be
> basically not letting you create corrupt data models which will
> explode later in unexpected and hard to determine ways. The reality is
> objects validation has been sneaking this in all along anyway.
> 
> The full monty approach
> ---
> 
> [composite:osapi_compute]
> use = call:nova.api.openstack.urlmap:urlmap_factory
> /: oscomputeversions
> /v1.1: openstack_compute_api_v2
> /v2: openstack_compute_api_v21
> # starting in Kilo the v21 implementation replaces the v2
> # implementation and is suggested that you use it as the default. If
> # this causes issues with your clients you can rollback to the
> # *frozen* v2 api by commenting out the above stanza and using the
> # following instead::
> # /v2: openstack_compute_api_v2
> # if rolling back to v2 fixes your issue please file a critical bug
> # at - https://bugs.launchpad.net/nova/+bugs
> 
> This would make the v2 endpoint the v21 implementation for new
> deploys. It would also make it easy for people to flip back if they
> hit an edge condition we didn't notice.
> 
> In functional testing we'd still test both v2 and v2.1
> 
> Tempest would move to v2.1 by default, and I think we should put an
> old v2 job on nova stable/kilo -> master to help us keep track of
> regressions.
> 
> The slow roll approach
> --
> 
> Ship the existing file (minus v3):
> 
> [composite:osapi_compute]
> use = call:nova.api.openstack.urlmap:urlmap_factory
> /: oscomputeversions
> /v1.1: openstack_compute_api_v2
> /v2: openstack_compute_api_v2
> /v2.1: openstack_compute_api_v21
> 
> The advantages here is that out of the box stuff keeps working. The
> dilema here is it's not really clear that we'll get people poking at
> v2.1 because it will be out of the main path. The point of the
> microversioning was to get folks onto that train soon because it falls
> back to existing API. And once we are convinced we're good we can
> deprecate out the old implementation.
> 
> Also, things like out of tree EC2 support requires v2.1, which is
> going to make deploys start relying on a /v2.1 endpoint that want EC2,
> so our options for grafting that back onto /v2 in the future are more
> limitted.
> 
> Decision Time
> =
> 
> Anyway, this is a decision we should make before freeze. The 'no
> decision' case gives us the slow roll. I think from an upstream
> perspective the full monty will probably serve us a little
> better. Especially with robust release notes that explain to people
> how to move their endpoints forward.
> 
> 
>   -Sean
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] what is our shipped paste.ini going to be for Kilo

[Christopher Yeoh]

On Tue, 17 Mar 2015 15:56:27 +1300
Robert Collins  wrote:

> On 17 March 2015 at 14:27, Ken'ichi Ohmichi 
> wrote:
> 
> >> I am worried about SDKs making requests that have additional JSON
> >> attributes that were previously ignored by v2, but will be
> >> considered invalid by the v2.1 validation code. If we were to just
> >> strip out the extra items, rather than error out the request (when
> >> you don't specify a microversion), I would be less worried about
> >> the transition. Maybe that what we do?
> >
> > Nice point.
> > That is a main difference in API behaviors between v2 and v2.1 APIs.
> > If SDKs pass additional JSON attributes to Nova API now, developers
> > need to fix/remove these attributes because that is a bug on SDKs
> > side.
> > These attributes are unused and meaningless, so some APIs of SDKs
> > would contain problems if passing this kind of attributes.
> >
> > Sometime it was difficult to know what are available attributes
> > before v2.1 API, so "The full monty approach" will clarify problems
> > of SDKs and make SDKs' quality better.
> >
> > Thanks
> > Ken Ohmichi
> 
> Better at the cost of forcing all existing users to upgrade just to
> keep using code of their own that already worked.
> 
> Not really 'better' IMO. Different surely.
> 
> We could (should) add Warning: headers to inform about this, but
> breaking isn't healthy IMO.


It'd be up to the operators, but there is always the option of simply
editing the paste.ini file so /v2 is again the produced by the old v2
code.


My main concern about v2 / v2.1 compatibility in practicce (rather than
just passing the same tempest and uniteststs which does work) is lack
of feedback. Probably don't exepct positive feedback in many cases but
we're not really getting negative feedback much either. I really would
appreciate people actually trying it more real world apps so we get a
better idea of the compatibility in areas of the code that don't have
good tempest coverage or have unitests which are incomplete.

Regards,

Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] what is our shipped paste.ini going to be for Kilo

[Christopher Yeoh]

On Mon, 16 Mar 2015 11:22:12 -0400
Russell Bryant  wrote:

> On 03/16/2015 11:08 AM, John Garbutt wrote:
> > While its not under Nova's control, I think we need to consider the
> > keystone catalog here.
> > 
> > It feels nice to have an explicit entry for v2.1, that people start
> > using once the client is updated to support microversions. DefCore
> > would eventually check for the presence of that entry.
> > 
> > Eventually, the v2 entry would be served by the v2.1 code, once we
> > are happy its not going to break too many folks, at least including
> > all the major SDKs. I hope thats during Liberty.
> 
> I'd really rather get away from any explicit entries in the keystone
> service catalog for API versions.  It seems kind of broken to me.  I
> think the service catalog should only have enough to point you to the
> APIs location.  Discovering and choosing versions should be done by
> the client and not embedded in the service catalog.

I agree, though the historical issue of why we have multiple keystone
entries for the Nova API is because the first one pointed to a specific
version rather than a generic place where a client can do version
discovery. We probably need to dig ourselves out of that hole first
(which is doable I think in conjunction with JSON-HOME which is an L
thing)

Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] is it possible to microversion a static class method?

[Christopher Yeoh]

So ultimately I think this is a style issue rather than a technical one. I
think there
are situations where one way looks clearer than another the other way does.
Sorry I can't get around to putting up a couple of examples,
ATM but to be clear there is no difference in the end result (no different
side effects etc)



On Mon, Mar 16, 2015 at 12:21 PM, Alex Xu  wrote:

>
>
> 2015-03-16 9:48 GMT+08:00 Alex Xu :
>
>>
>>
>> 2015-03-13 19:10 GMT+08:00 Sean Dague :
>>
>>> On 03/13/2015 02:55 AM, Chris Friesen wrote:
>>> > On 03/12/2015 12:13 PM, Sean Dague wrote:
>>> >> On 03/12/2015 02:03 PM, Chris Friesen wrote:
>>> >>> Hi,
>>> >>>
>>> >>> I'm having an issue with microversions.
>>> >>>
>>> >>> The api_version() code has a comment saying "This decorator MUST
>>> appear
>>> >>> first (the outermost decorator) on an API method for it to work
>>> >>> correctly"
>>> >>>
>>> >>> I tried making a microversioned static class method like this:
>>> >>>
>>> >>>  @wsgi.Controller.api_version("2.4")  # noqa
>>> >>>  @staticmethod
>>> >>>  def _my_func(req, foo):
>>> >>>
>>> >>> and pycharm highlighted the api_version decorator and complained that
>>> >>> "This decorator will not receive a callable it may expect; the
>>> built-in
>>> >>> decorator returns a special object."
>>> >>>
>>> >>> Is this a spurious warning from pycharm?  The pep8 checks don't
>>> >>> complain.
>>> >>>
>>> >>> If I don't make it static, then pycharm suggests that the method
>>> could
>>> >>> be static.
>>> >>
>>> >> *API method*
>>> >>
>>> >> This is not intended for use by methods below the top controller
>>> level.
>>> >> If you want conditionals lower down in your call stack pull the
>>> request
>>> >> version out yourself and use that.
>>> >
>>> > Both the original spec and doc/source/devref/api_microversions.rst
>>> > contain text talking about decorating a private method.  The latter
>>> > gives this example:
>>> >
>>> > @api_version("2.1", "2.4")
>>> > def _version_specific_func(self, req, arg1):
>>> > pass
>>> >
>>> > @api_version(min_version="2.5") #noqa
>>> > def _version_specific_func(self, req, arg1):
>>> > pass
>>> >
>>> > def show(self, req, id):
>>> >  common stuff 
>>> > self._version_specific_func(req, "foo")
>>> >  common stuff 
>>> >
>>> > It's entirely possible that such a private method might not need to
>>> > reference "self", and could therefore be static, so I think it's a
>>> valid
>>> > question.
>>>
>>> That's a doc bug, we should change it.
>>>
>>
>>
>> Actually it is not a bug. It's controversial point in the spec, but
>> finally that was keep in the spec.
>>
>> http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/api-microversions.html
>>
>> The discussion at line 268
>> https://review.openstack.org/#/c/127127/7/specs/kilo/approved/api-microversions.rst
>>
>
> Submit a patch for devref https://review.openstack.org/164555  Let see
> whether we can get agreement
>
>
>>
>>
>>>
>>> -Sean
>>>
>>> --
>>> Sean Dague
>>> http://dague.net
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] how safe is it to change NoAuthMiddlewareBase?

[Christopher Yeoh]

On Sat, 28 Feb 2015 09:51:27 -0700
Jay Pipes  wrote:

> On 02/26/2015 04:27 AM, Sean Dague wrote:
> > In trying to move the flavor manage negative tests out of Tempest
> > and into the Nova functional tree, I ran into one set of tests
> > which are permissions checking. Basically that a regular user isn't
> > allowed to do certain things.
> >
> > In (nearly) all our tests we use auth_strategy=noauth which takes
> > you to NoAuthMiddlewareBase instead of to keystone. That path makes
> > you an admin regardless of what credentials you send in -
> > https://github.com/openstack/nova/blob/master/nova/api/openstack/auth.py#L56-L59
> >
> > What I'd like to do is to change this so that if you specify
> > user_id='admin' then is_admin is set true, and it's not true
> > otherwise.
> >
> > That has a bunch of test fall out, because up until this point most
> > of the test users are things like 'fake', which would regress to
> > non admin. About 25% of the api samples tests fail in such a
> > change, so they would need to be fixed.
> 
> Taking a step back... what exactly is the purpose of the API samples 
> "functional tests"? If the purpose of these tests has anything to do 
> with validating some policy thing, then I suppose it's worth changing 
> the auth middleware to support non-adminness. But, 

Historically I think its been a couple of reasons
- to generate api samples for documentation purposes
- to do more thorough testing of Nova that previously should have
gone into tempest (but everything is moving back now right?) but writing
tempests tests has been seen as too hard a postponed until after
the Nova change has merged and then the intent has been lost.

>I don't think the
> API samples test purpose has anything to do with that (I think the
> purpose of the API samples tests is fuzzy, at best, actually). So,
> I'd just leave them as-is and not change anything at all.

If we're moving stuff over from tempest to nova we definitely need to
keep tracking of what has been done and what we need to do.
Eg we need to find out what state we're in first. Definitely have picked
up lots of issues in the past with the functional tests that
the unitests have missed.

Chris
> 
> Best,
> -jay
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] need input on possible API change for bug #1420848

[Christopher Yeoh]

FWIW I think we need to consider that the API is completely froxen for the
V2 API (so this freeze does not apply to v2.1 microversions) except under
very serious circumstances and only very high priority bug fixes and only
apply this to a suitable microversion bump. We really want to get rid of
the V2 API code asap anyway.  You can after all request a version number of
a per method basis as long as you are talking to v2.1. So the only forced
upgrade is the v2->v2.1 transition and those apis should be identical

On Fri, Mar 13, 2015 at 12:02 AM, Christopher Yeoh 
wrote:

>
>
>
> On Thu, Mar 12, 2015 at 11:19 PM, Christopher Yeoh 
> wrote:
>
>> On Wed, 11 Mar 2015 09:32:11 -0600
>> Chris Friesen  wrote:
>>
>> >
>> > Hi,
>> >
>> > I'm working on bug #1420848 which addresses the issue that doing a
>> > "service-disable" followed by a "service-enable" against a "down"
>> > compute node will result in the compute node going "up" for a while,
>> > possibly causing delays to operations that try to schedule on that
>> > compute node.
>> >
>> > The proposed solution is to add a new "reported_at" field in the DB
>> > schema to track when the service calls _report_state().
>> >
>> > The backend is straightforward, but I'm trying to figure out the best
>> > way to represent this via the REST API response.
>> >
>> > Currently we response includes an "updated_at" property, which maps
>> > directly to the auto-updated "updated_at" field in the database.
>> >
>> > Would it be acceptable to just put the "reported_at" value (if it
>> > exists) in the "updated_at" property?  Logically the "reported_at"
>> > value is just a determination of when the service updated its own
>> > state, so an argument could be made that this shouldn't break
>> > anything.
>> >
>>
>>
>> So i think this is the critical point here is this a backwards
>> compatibly API change or not. Would reporing reported_at in updated_at
>> cause *anyone* any pain. For this reason I think it has to go through
>> as a nova spec (and if you think its not going to cause pain get some
>> people from the mailing list +1 it as backwards API change because it
>> always has been a bug. If that is the conculsion and you just reuse
>> updated_at
>
> then the procedure would be:
>
>>
>> - Add it to v2 and no v2 extension required
>> - Add it to v2.1 without an extension
>> - No change required to in terms of microversions because it is lready
>>   in the base v2.1 code
>>
>> If you go the reported_at route the and there no changed to updated_at
>>
> but the fix is consiered a new feature rather than a bug fix then I think
> we should seriously consider if it should be fixed in V2 at all because the
> v2 api is basically frozen and we can just add it as a microversion (don't
> even need to to support it in v2.1), just  api microversion
>
> In which case the documents that Kevin pointed to should help - if you
> have any problems catch me on irc or on return email
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] need input on possible API change for bug #1420848

[Christopher Yeoh]

On Thu, Mar 12, 2015 at 11:19 PM, Christopher Yeoh 
wrote:

> On Wed, 11 Mar 2015 09:32:11 -0600
> Chris Friesen  wrote:
>
> >
> > Hi,
> >
> > I'm working on bug #1420848 which addresses the issue that doing a
> > "service-disable" followed by a "service-enable" against a "down"
> > compute node will result in the compute node going "up" for a while,
> > possibly causing delays to operations that try to schedule on that
> > compute node.
> >
> > The proposed solution is to add a new "reported_at" field in the DB
> > schema to track when the service calls _report_state().
> >
> > The backend is straightforward, but I'm trying to figure out the best
> > way to represent this via the REST API response.
> >
> > Currently we response includes an "updated_at" property, which maps
> > directly to the auto-updated "updated_at" field in the database.
> >
> > Would it be acceptable to just put the "reported_at" value (if it
> > exists) in the "updated_at" property?  Logically the "reported_at"
> > value is just a determination of when the service updated its own
> > state, so an argument could be made that this shouldn't break
> > anything.
> >
>
>
> So i think this is the critical point here is this a backwards
> compatibly API change or not. Would reporing reported_at in updated_at
> cause *anyone* any pain. For this reason I think it has to go through
> as a nova spec (and if you think its not going to cause pain get some
> people from the mailing list +1 it as backwards API change because it
> always has been a bug. If that is the conculsion and you just reuse
> updated_at

then the procedure would be:

>
> - Add it to v2 and no v2 extension required
> - Add it to v2.1 without an extension
> - No change required to in terms of microversions because it is lready
>   in the base v2.1 code
>
> If you go the reported_at route the and there no changed to updated_at
>
but the fix is consiered a new feature rather than a bug fix then I think
we should seriously consider if it should be fixed in V2 at all because the
v2 api is basically frozen and we can just add it as a microversion (don't
even need to to support it in v2.1), just  api microversion

In which case the documents that Kevin pointed to should help - if you have
any problems catch me on irc or on return email

>
> > Otherwise, by my reading of
> > "
> https://wiki.openstack.org/wiki/APIChangeGuidelines#Generally_Considered_OK
> "
> > it seems like if I wanted to add a new "reported_at" property I would
> > need to do it via an API extension.
> >
> > Anyone have opinions?
> >
> > Chris
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Christopher Yeoh]

On Mon, 09 Mar 2015 16:14:21 -0400
Sean Dague  wrote:

> On 03/09/2015 03:37 PM, Jay Pipes wrote:
> > On 03/08/2015 08:10 AM, Alex Xu wrote:
> >> Thanks for Jay point this out! If we have agreement on this and
> >> document it, that will be great for guiding developer how to add
> >> new API.
> >>
> >> I know we didn't want extension for API. But I think we still
> >> need modularity. I don't think we should put everything in a single
> >> file, that file will become huge in the future and hard to
> >> maintenance.
> > 
> > I don't think everything should be in a single file either. In fact,
> > I've never advocated for that.
> > 
> >> We can make the 'extension' not configurable. Replace 'extension'
> >> with another name, deprecate the extension info api int he
> >> future But that is not mean we should put everything in a file.
> > 
> > I didn't say that in my email. I'm not sure where you got the
> > impression I want to put everything in one file?
> > 
> >> For modularity, we need define what should be in a separated
> >> module(it is extension now.) There are three cases:
> >>
> >> 1. Add new resource
> >>  This is totally worth to put in a separated module.
> > 
> > Agreed.
> > 
> >> 2. Add new sub-resource
> >>  like server-tags, I prefer to put in a separated module, I
> >> don't think put another 100 lines code in the servers.py is good
> >> choice.
> > 
> > Agreed, which is exactly what I said in my email:
> > 
> > "Similarly, new microversion API functionality should live in a
> > module, as a top-level (or subcollection) Controller in
> > /nova/api/openstack/compute/, and should not be in the
> > /nova/api/openstack/compute/plugins/ directory. Why? Because it's
> > not a plugin."
> > 

Ok so I'm getting confused about what we're disagreeing about then.

However, the first microversion change
https://review.openstack.org/#/c/140313/32 is one case where we didn't
need to create a new extension, relying only on microversions to add a
new parameter to the response, whereas server tags does add a new
conroller (os-server-tags) which is non trivia so I think it does. Do
we disagree about that?

btw in a situation now where (I think) we are saying we are not going
to do any work for 3rd parties to add their own API plugins and have a
well planned API we don't need the prefixes as we don't need the prefix
on parameter names as there won't be name clashes without us realising
during testing. And we certainly don't need the os- prefix is the
plugin alias, but we do neeed them unique across the API I believe
because of the way we store information about them.

> >> 3. extend attributes and methods for a existed resource
> >> like add new attributes for servers, we can choice one of
> >> existed module to put it in. Just like this patch
> >> https://review.openstack.org/#/c/155853/
> >> But for servers-tags, it's sub-resource, we can put it in
> >> its-own module.
> > 
> > Agreed, and that's what I put in my email.
> > 
> >> If we didn't want to support extension right now, we can begin
> >> from not show servers-tags in extension info API first. That means
> >> extension info is freeze now. We deprecated the extension info api
> >> in later version.


It can be surpressed from /extensions by adding it to the suppress list
in the extensions code. Probably a good idea to stop v2.1+microversion
code REST API users not accidentally relying on it.
> > 
> > I don't understand what you're saying here. Could you elaborate?
> > What I am asking for is for new functionality (like the server-tags
> > subcollection resource), just add a new module called
> > /nova/api/openstack/compute/server_tags.py, create a Controller
> > object in that file with the new server tags resource, and don't
> > use any of the API extensions framework whatsoever.
> > 
> > In addition to that, for the changes to the main GET
> > /servers/{server_id} resource, use microversions to decorate the
> > /nova/api/openstack/compute/servers.py.Controller.show() method for
> > 2.4 and add a "tags" key to the dict (note: NOT a
> > "os-server-tags:tags" key) returned by GET /servers/{server_id}. No
> > use of API extensions needed.
>


So I that's doabe but I think if we compared to the two wsgi.extends
is cleaner and less code and we have to have the separate module file
anyway for the controller. Can discuss this more later

Incidentally as has been mentioned before we need new names for the
API and where the files need to live needs cleaning up. For example v3
and v2.1 needs to be worked out of the directory paths.This clenaup not
on involves but directly  affects the all the related unit and
functional tests. Nor should we have contrib or should v2 live directly
in in nova/api/openstack/compute. But we also need a name for v2.1
microversions and should spend some time on that (does api modules work
for anyone?)

I think this dicussion indicates we should start with a bit of planning
first rather than just jump in and sta

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Christopher Yeoh]

On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt  wrote:

> +1
>
> Please could you submit a dev ref for this?
>
> We can argue on the review, a bit like this one:
>
> https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst
>
> I think it'd also be a good idea to add a testcase (use test_plugins
directory where you can
define your own controller which is never plubished) for each example so
they don't get out of date

Regards,

Chris
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Christopher Yeoh]

On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt  wrote:

> Hi,
>
> I think I agree with Jay here, but let me explain...
>
> On 8 March 2015 at 12:10, Alex Xu  wrote:
> > Thanks for Jay point this out! If we have agreement on this and document
> it,
> > that will be great for guiding developer how to add new API.
>
> +1
>
> Please could you submit a dev ref for this?
>
> We can argue on the review, a bit like this one:
>
> https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst
>
> > For modularity, we need define what should be in a separated module(it is
> > extension now.) There are three cases:
> >
> > 1. Add new resource
> > This is totally worth to put in a separated module.
>
> +1
>
> > 2. Add new sub-resource
> > like server-tags, I prefer to put in a separated module, I don't
> think
> > put another 100 lines code in the servers.py is good choice.
>
> -1
>
> I hate the idea of show instance extension code for version 2.4 living
> separately to the rest of the instance show logic, when it really
> doesn't have to.
>
> It feels too heavyweight in its current form.
>
>
If the only thing server-tags did was to add a parameter then we wouldn't
need a new extension,
but its not, it adds another resource with associated actions


> Maybe we need a more modular way of expressing the extension within
> the same file?
>
>
I think servers.py is simply to big. Its much harder to read and debug than
any other plugin just because of its size - or
maybe I just need a 50" monitor :) I'd rather ensure functionality common
server-tags and the API is kept together rather than
spread through servers.py



> > 3. extend attributes and methods for a existed resource
> >like add new attributes for servers, we can choice one of existed
> module
> > to put it in. Just like this patch
> https://review.openstack.org/#/c/155853/
>
> +1
>
> I wish it was easier to read, but I hope thats fixable long term.
>
> > 2015-03-08 8:31 GMT+08:00 Jay Pipes :
> >> Now that microversions have been introduced to the Nova API (meaning we
> >> can now have novaclient request, say, version 2.3 of the Nova API using
> the
> >> special X-OpenStack-Nova-API-Version HTTP header), is there any good
> reason
> >> to require API extensions at all for *new* functionality.
>
> As above, a new "resource" probably should get a new "plugins/v3" module
> right?
>
> It feels (at worst) borderline in the os-server-tags case, due to the
> extra actions.
>
> >> What is the point of creating a new "plugin"/API extension for this new
> >> functionality? Why can't we just modify the
> >> nova/api/openstack/compute/server.py Controller.show() method and
> decorate
> >> it with a 2.4 microversion that adds a "tags" attribute to the returned
> >> server dictionary?
> >>
> >> Similarly, new microversion API functionality should live in a module,
> as
> >> a top-level (or subcollection) Controller in
> /nova/api/openstack/compute/,
> >> and should not be in the /nova/api/openstack/compute/plugins/ directory.
> >> Why? Because it's not a plugin.
>
> Everything is a "plugin" in v3, no more distinction between core vs
> plugin. It needs renaming really.
>
> It should look just like servers, I guess, which is a top level item:
>
> https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/plugins/v3/servers.py
>
> >> Why are we continuing to use these awkward, messy, and cumbersome API
> >> extensions?
>
> We certainly should never be forced to add an extension to advertise
> new functionality anymore.
>
> Its a big reason why I want to see the API micro-versions succeed.
>

Yep, there is I think no reason except to support /extensions for now and I
don't really think its worth having
two entry points, one for modules which will appear in /extensions and one
for modules which won't appear
in /extensioins. The overhead is low. We should warn v2.1+ users to ignore
/extensions unless they are legacy v2 api users
and they should remove their use of it anyway as soon as they get off v2.1.
They key to dumping it all is
when people tell us v2.1 really is behaving just like v2 so we can remove
the old v2 code and then later have a microversion that
doesn't support /extensions. I hope all the json-home stuff is in by then
:-)

>
> >> Please, I am begging the Nova core team. Let us stop this madness. No
> more
> >> API extensions.
>
> Lets try get something agreed in devref, so we are ready to go when
> Liberty opens.
>
> It would be nice to look at ways to fold back the existing extensions
> into the main code. I know there are v2.0 compatibility issues there,
> but I think/hope thats mostly cosmetic at this point.
>
>
Yea we already did a lot of that in v3 and had to separate  some of them
out again for v2.1 (argh!). Others we have just faked (eg you load module
"X" you get module "Y" for free which doesn't really exist anymore - but
only for those that we were very sure that the extension only existed to
notify users that s

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Christopher Yeoh]

On Mon, Mar 9, 2015 at 9:35 PM, Sean Dague  wrote:

> On 03/07/2015 07:31 PM, Jay Pipes wrote:
> > Hi Stackers,
> >
> > Now that microversions have been introduced to the Nova API (meaning we
> > can now have novaclient request, say, version 2.3 of the Nova API using
> > the special X-OpenStack-Nova-API-Version HTTP header), is there any good
> > reason to require API extensions at all for *new* functionality.
> >
> > Sergey Nikitin is currently in the process of code review for the final
> > patch that adds server instance tagging to the Nova API:
> >
> > https://review.openstack.org/#/c/128940
> >
> > Unfortunately, for some reason I really don't understand, Sergey is
> > being required to create an API extension called "os-server-tags" in
> > order to add the server tag functionality to the API. The patch
> > implements the 2.4 Nova API microversion, though, as you can see from
> > this part of the patch:
> >
> >
> https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py
> >
> >
> > What is the point of creating a new "plugin"/API extension for this new
> > functionality? Why can't we just modify the
> > nova/api/openstack/compute/server.py Controller.show() method and
> > decorate it with a 2.4 microversion that adds a "tags" attribute to the
> > returned server dictionary?
> >
> >
> https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369
> >
> >
> > Because we're using an API extension for this new server tags
> > functionality, we are instead having the extension "extend" the server
> > dictionary with an "os-server-tags:tags" key containing the list of
> > string tags.
> >
> > This is ugly and pointless. We don't need to use API extensions any more
> > for this stuff.
> >
> > A client knows that server tags are supported by the 2.4 API
> > microversion. If the client requests the 2.4+ API, then we should just
> > include the "tags" attribute in the server dictionary.
> >
> > Similarly, new microversion API functionality should live in a module,
> > as a top-level (or subcollection) Controller in
> > /nova/api/openstack/compute/, and should not be in the
> > /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a
> > plugin.
> >
> > Why are we continuing to use these awkward, messy, and cumbersome API
> > extensions?
> >
> > Please, I am begging the Nova core team. Let us stop this madness. No
> > more API extensions.
>
> Agreed, the current extensions list exists to explain the base v2
> functionality. I think we should consider that frozen and deprecated as
> of v2.1 as we have a better way to express features.
>
> -Sean
>
>

So I think we can a microversion ASAP to remove support for /extensions.
Obviously we'll need to keep the actual code
to support v2.1 for quite a while though.

I think we still want some fields in the controller like we do because we
want to automate JSON-HOME/Schema stuff (maybe not sure)


> --
> Sean Dague
> http://dague.net
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Christopher Yeoh]

Hi,

Apologies for the slow reply, long weekend because of a public holiday over
here. I'm probably going to end up repeating part of what
Alex has mentioned as well.

So the first thing I think we want to distinguish between plugins being a
REST API user or operator concept and it being
 a tool developers use as a framework to support the Nova REST API. As I've
mentioned before I've no problem with the feature set of the
API being fixed (per microversion) across all Nova deployments. Get back to
me when we have consensus on that and its trivial to
implement and we'll no longer have the concept of core and extension/plugin.

But plugin like implementations using Stevedore as a tool for developers to
keep good modularity has proven to be very useful to keep complexity level
lower and interactions between modules much clearer. servers.py is an
example of this where in v2 I think we have/had the most complex method and
even with all the fix up work which has been done on it it is still very
complicated to understand.


On Sun, Mar 8, 2015 at 11:01 AM, Jay Pipes  wrote:

> Hi Stackers,
>
> Now that microversions have been introduced to the Nova API (meaning we
> can now have novaclient request, say, version 2.3 of the Nova API using the
> special X-OpenStack-Nova-API-Version HTTP header), is there any good reason
> to require API extensions at all for *new* functionality.
>
> Sergey Nikitin is currently in the process of code review for the final
> patch that adds server instance tagging to the Nova API:
>
> https://review.openstack.org/#/c/128940
>
> Unfortunately, for some reason I really don't understand, Sergey is being
> required to create an API extension called "os-server-tags" in order to add
> the server tag functionality to the API. The patch implements the 2.4 Nova
> API microversion, though, as you can see from this part of the patch:
>
> https://review.openstack.org/#/c/128940/43/nova/api/
> openstack/compute/plugins/v3/server_tags.py
>
> What is the point of creating a new "plugin"/API extension for this new
> functionality? Why can't we just modify the 
> nova/api/openstack/compute/server.py
> Controller.show() method and decorate it with a 2.4 microversion that adds
> a "tags" attribute to the returned server dictionary?
>
>
Actually I think it does more than just add extra reponse information:
- it adds extra tags parameter to show
  - it doesn't add it to index, but it probably should add the response
information to detail to be consistent with the rest of the API
- It adds a new resource /servers/server_id/tags
   - with create, delete and delete all supported. I don't think that these
belong in servers.py



> https://github.com/openstack/nova/blob/master/nova/api/
> openstack/compute/servers.py#L369
>
> Because we're using an API extension for this new server tags
> functionality, we are instead having the extension "extend" the server
> dictionary with an "os-server-tags:tags" key containing the list of string
> tags.
>
> This is ugly and pointless. We don't need to use API extensions any more
> for this stuff.
>
>
So we had a prefix rule originally in V2 to allow for extensions and
guarantee no name clashes. I'd be happy removing this requirement, even
removing old ones as long as we have consensus.


> A client knows that server tags are supported by the 2.4 API microversion.
> If the client requests the 2.4+ API, then we should just include the "tags"
> attribute in the server dictionary.
>
> Similarly, new microversion API functionality should live in a module, as
> a top-level (or subcollection) Controller in /nova/api/openstack/compute/,
> and should not be in the /nova/api/openstack/compute/plugins/ directory.
> Why? Because it's not a plugin.
>
> So I don't see how that changes whether we're using plugins (from a user
point of view) or not. The good news for you is that
there is fixing the shambles of a directory structure for the api is on the
list of things to do, it just wasn't a high prioirty things for us in Kilo,
get v2.1 and microversions out. For example, we have v3 in the directory
path as well for historical reasons and we also have a contrib directory in
compute and none of those are really "contrib" now either.  Now the
nova/api/openstack/compute/ directory where you want to put all the v2
microversions code is currently full of v2 core code already.  It just
makes more sense to me to wait unti the old v2 core code can be removed
because the v2.1 api is considered equivalent and then move the v2.1
microversions code into its final place , rather than a shuffle now to move
the old v2 code (along with all the changes need to the unittests) and then
just have to delete it again not much longer.





> Why are we continuing to use these awkward, messy, and cumbersome API
> extensions?
>
> Please, I am begging the Nova core team. Let us stop this madness. No more
> API extensions.
>
>
It is still not clear to me exactly what you mean by use of an extension.
None of us ours 

Re: [openstack-dev] [nova] what's the merge plan for current proposed microversions?

[Christopher Yeoh]

On Wed, Mar 4, 2015 at 9:51 PM, Alexandre Levine 
wrote:

> Christopher,
>
> Does this
>
> "So the plan for assignment of microversion api numbers is the same as
> what we currently do for db migration changes - take the next one
> knowing that you may need to rebase if someone else merges before you"
>
> mean that I should put 2.2 in my review for now instead of 2.4?
>
>
I don't think that'd be worth it because in this case as its the first
microversion we've already decided to merge
https://review.openstack.org/140313
first and you'll need to rebase to at least 2.3. I think the reason I
recommended 2.4 to you was that https://review.openstack.org/128940 was the
other patch identified as
a possible good candidate for being the first microversion but in the end
wasn't selected so it ended up with 2.3. From a quick look at 128940  I
think it might have spec approval
issues thoughso if your patch is ready when 140313 merges you might end up
with 2.3.


> Other suggestion would be to pack several simultaneously incoming changes
> into one microversion. Maybe spawn them once a week, or once a couple of
> weeks, or even with longer period, depending on the amount of incoming
> changes. For example, wouldn't it be convenient for clients to acquire all
> of the accumulated changes in one microversion (2.2 as I understand)
> without needing to understand which one came with what number? To clarify,
> I'm suggesting to pass reviews for all of the hanging API changes against
> 2.2 version.
>
>

So I think its probably better to keep the number of changes small per
microversion. Though I have also suggested in the past that very minor
changes in the same such as formatting etc be fixed if we're making api
chnges in the same area anyway and clients will be forced to modify what
they do regardless. However bundling a lot of api changes in one
microversion will for CD we'd need them to be enabled all at once meaning
we'd either need to merge them into one patch or have an additional patch
which just enables say 2.2 once all the dependent patches are merged. This
increases the probability that one delayed patch will delay a bunch of
others whereas now whoever is ready the first will merge first..

It someone has experience from db migrations that they think would work
well, please let us know!

Regards,

Chris

Best regards,
>   Alex Levine
>
>
> On 3/4/15 11:44 AM, Christopher Yeoh wrote:
>
>> On Tue, 03 Mar 2015 10:28:34 -0500
>> Sean Dague  wrote:
>>
>>  On 03/03/2015 10:24 AM, Claudiu Belu wrote:
>>>
>>>> Hello.
>>>>
>>>> I've talked with Christopher Yeoh yesterday and I've asked him
>>>> about the microversions and when will they be able to merge. He
>>>> said that for now, this commit had to get in before any other
>>>> microversions: https://review.openstack.org/#/c/159767/
>>>>
>>>> He also said that he'll double check everything, and if everything
>>>> is fine, the first microversions should be getting in soon after.
>>>>
>>>> Best regards,
>>>>
>>>> Claudiu Belu
>>>>
>>> I just merged that one this morning, so hopefully we can dislodge.
>>>
>>
>> So just before we merged the the keypairs microversion change someone
>> enabled response schema tests which showed some further problems. They
>> have now all been fixed but https://review.openstack.org/#/c/161112/
>> which needs just one more +2. After that the first api change which uses
>> microversions https://review.openstack.org/#/c/140313/ can merge (has
>> 3 +2's just needs the v2.1 fix first.
>>
>>  
>>>> From: Alexandre Levine [alev...@cloudscaling.com]
>>>> Sent: Tuesday, March 03, 2015 4:22 PM
>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>> Subject: Re: [openstack-dev] [nova] what's the merge plan for
>>>> current proposed microversions?
>>>>
>>>> Bump.
>>>>
>>>> I'd really appreciate some answers to the question Sean asked. I
>>>> still have the 2.4 in my review (the very one Sean mentioned) but
>>>> it seems that it might not be the case.
>>>>
>>>> Best regards,
>>>> Alex Levine
>>>>
>>>> On 3/2/15 2:30 PM, Sean Dague wrote:
>>>>
>>>>> This change for the additional attributes for ec2 looks like it's
>>>>> basically ready to go, except it has the wrong microversion on it
>>>>> (as they anticipated 

Re: [openstack-dev] [nova] what's the merge plan for current proposed microversions?

[Christopher Yeoh]

On Tue, 03 Mar 2015 10:28:34 -0500
Sean Dague  wrote:

> On 03/03/2015 10:24 AM, Claudiu Belu wrote:
> > Hello.
> > 
> > I've talked with Christopher Yeoh yesterday and I've asked him
> > about the microversions and when will they be able to merge. He
> > said that for now, this commit had to get in before any other
> > microversions: https://review.openstack.org/#/c/159767/
> > 
> > He also said that he'll double check everything, and if everything
> > is fine, the first microversions should be getting in soon after.
> > 

> > Best regards,
> > 
> > Claudiu Belu
> 
> I just merged that one this morning, so hopefully we can dislodge.


So just before we merged the the keypairs microversion change someone
enabled response schema tests which showed some further problems. They
have now all been fixed but https://review.openstack.org/#/c/161112/
which needs just one more +2. After that the first api change which uses
microversions https://review.openstack.org/#/c/140313/ can merge (has
3 +2's just needs the v2.1 fix first.

> 
> > 
> > 
> > From: Alexandre Levine [alev...@cloudscaling.com]
> > Sent: Tuesday, March 03, 2015 4:22 PM
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: Re: [openstack-dev] [nova] what's the merge plan for
> > current proposed microversions?
> > 
> > Bump.
> > 
> > I'd really appreciate some answers to the question Sean asked. I
> > still have the 2.4 in my review (the very one Sean mentioned) but
> > it seems that it might not be the case.
> > 
> > Best regards,
> >Alex Levine
> > 
> > On 3/2/15 2:30 PM, Sean Dague wrote:
> >> This change for the additional attributes for ec2 looks like it's
> >> basically ready to go, except it has the wrong microversion on it
> >> (as they anticipated the other changes landing ahead of them) -
> >> https://review.openstack.org/#/c/155853
> >>
> >> What's the plan for merging the outstanding microversions? I
> >> believe we're all conceptually approved on all them, and it's an
> >> important part of actually moving forward on the new API. It seems
> >> like we're in a bit of a holding pattern on all of them right now,
> >> and I'd like to make sure we start merging them this week so that
> >> they have breathing space before the freeze.
> >>

So the plan for assignment of microversion api numbers is the same as
what we currently do for db migration changes - take the next one
knowing that you may need to rebase if someone else merges before you.
Other suggestions welcome but will have to follow the requirement that
they always merge in version order.



> >>   -Sean
> >>
> > 
> > 
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> 
> 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Temporary freeze on API changes

[Christopher Yeoh]

Hi,

The Nova API subgroup is planning on releasing V2.1 on Monday (changing its
status from experimental to CURRENT) and merging the first patch which uses
microversions on top of 2.1 on the Wednesday.

In the meantime we'd appreciate it if reviewers did not +A any patchset
which changes the REST API (as it potentially means we'd have to sync
changes to 2.1 as well. Any future api changes should only be applied to
the v2.1/v3 tree and use the microversions mechanism.

When https://review.openstack.org/#/c/140313/ is merged, that will signal
that microversions
is enabled and we are open for patches to v2.1 microversions. At this point
no changes should be accepted to the old v2 api code and anything to the
v2.1 code base should be done using the microversions mechanism.

There is devref docs on how to use microversions either merged or working
its way through gerrit.

Regards,

Chris
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Nova API meeting

[Christopher Yeoh]

Hi,

Just a reminder that the weekly Nova API meeting is being held tomorrow
Friday UTC . 

We encourage cloud operators and those who use the REST API such as
SDK developers and others who and are interested in the future of the
API to participate.

In other timezones the meeting is at:

EST 20:00 (Thu)
Japan 09:00 (Fri)
China 08:00 (Fri)
ACDT 10:30 (Fri)

The proposed agenda and meeting details are here: 

https://wiki.openstack.org/wiki/Meetings/NovaAPI

Please feel free to add items to the agenda. 


We will also be discussing when to release v2.1 and microversions. I'm
going to propose that v2.1 becomes non experimental on Monday and
microversions is enabled with the first api change to use it on
Wednesday.

Please yell here or at the meeting if you think thats a bad idea.
Note that the old v2 code is going to remain the default on /v2 so you
still need to "opt-in" to v2.1 and microversioned changes will only
affect you if you send the appropriate header

Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Outcome of the nova FFE meeting for Kilo

[Christopher Yeoh]

On Wed, Feb 18, 2015 at 6:18 AM, Matt Riedemann 
wrote:

>
>
> On 2/16/2015 9:57 PM, Jay Pipes wrote:
>
>> Hi Mikal, sorry for top-posting. What was the final decision regarding
>> the instance tagging work?
>>
>> Thanks,
>> -jay
>>
>> On 02/16/2015 09:44 PM, Michael Still wrote:
>>
>>> Hi,
>>>
>>> we had a meeting this morning to try and work through all the FFE
>>> requests for Nova. The meeting was pretty long -- two hours or so --
>>> and we did in in the nova IRC channel in an attempt to be as open as
>>> possible. The agenda for the meeting was the list of FFE requests at
>>> https://etherpad.openstack.org/p/kilo-nova-ffe-requests
>>>
>>> I recognise that this process is difficult for all, and that it is
>>> frustrating when your FFE request is denied. However, we have tried
>>> very hard to balance distractions from completing priority tasks and
>>> getting as many features into Kilo as possible. I ask for your
>>> patience as we work to finalize the Kilo release.
>>>
>>> That said, here's where we ended up:
>>>
>>> Approved:
>>>
>>>  vmware: ephemeral disk support
>>>  API: Keypair support for X509 public key certificates
>>>
>>> We were also presented with a fair few changes which are relatively
>>> trivial (single patch, not very long) and isolated to a small part of
>>> the code base. For those, we've selected the ones with the greatest
>>> benefit. These ones are approved so long as we can get the code merged
>>> before midnight on 20 February 2015 (UTC). The deadline has been
>>> introduced because we really are trying to focus on priority work and
>>> bug fixes for the remainder of the release, so I want to time box the
>>> amount of distraction these patches cause.
>>>
>>> Those approved in this way are:
>>>
>>>  ironic: Pass the capabilities to ironic node instance_info
>>>  libvirt: Nova vif driver plugin for opencontrail
>>>  libvirt: Quiescing filesystems with QEMU guest agent during image
>>> snapshotting
>>>  libvirt: Support vhost user in libvirt vif driver
>>>  libvirt: Support KVM/libvirt on System z (S/390) as a hypervisor
>>> platform
>>>
>>> It should be noted that there was one request which we decided didn't
>>> need a FFE as it isn't feature work. That may proceed:
>>>
>>>  hyperv: unit tests refactoring
>>>
>>> Finally, there were a couple of changes we were uncomfortable merging
>>> this late in the release as we think they need time to "bed down"
>>> before a release we consider stable for a long time. We'd like to see
>>> these merge very early in Liberty:
>>>
>>>  libvirt: use libvirt storage pools
>>>  libvirt: Generic Framework for Securing VNC and SPICE
>>> Proxy-To-Compute-Node Connections
>>>
>>> Thanks again to everyone with their patience with our process, and
>>> helping to make Kilo an excellent Nova release.
>>>
>>> Michael
>>>
>>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
>> unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> There are notes in the etherpad,
>
> https://etherpad.openstack.org/p/kilo-nova-ffe-requests
>
> but I think we wanted to get cyeoh and Ken'ichi's thoughts on the v2
> and/or v2.1 question about the change, i.e. should it be v2.1 only with
> microversions or if that is going to block it, is it fair to keep out the
> v2 change that's already in the patch?
>
>
So if it can be fully merged by end of week I'm ok with it going into v2
and v2.1. Otherwise I think it needs to wait for microversions. I'd like to
see v2.1 enabled next Monday (I don't want it go in just before a weekend).
And the first microversion change (which is ready to go) a couple of days
after). And we want a bit of an API freeze while that is happening.

Chris




> --
>
> Thanks,
>
> Matt Riedemann
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Feature Freeze Exception request for x509 keypairs

[Christopher Yeoh]

I'm happy to sponsor this. I've reviewed all the patches as well and as
Claudiu mentions we have this lined up as being the first api change to use
microversions

Regards,

Chris

On Thu, Feb 12, 2015 at 10:50 PM, Claudiu Belu  wrote:

>
>  Hello.
>
> I would like to ask for a FFE for the x509 keypairs blueprint:
> https://blueprints.launchpad.net/nova/+spec/keypair-x509-certificates
>
> This blueprint is split up into 3 commits:
>
> [1] Database migration: previously merged, but had to be reverted because
> of a small issue. Everything is fixed, original reverter Johannes Erdfelt
> gave his +1, currently the commit has a +2.
> https://review.openstack.org/#/c/150800/
>
> [2] Nova-API change: It uses the microversioning API and it has been
> decided to be the first microversioning commit, since it is closest to
> merge. Christopher Yeoh reviewed helped with this commit.
> https://review.openstack.org/#/c/140313/
>
> [3] X509 keypair implementation: Simple commit, it had a +2 on a previous
> commit. https://review.openstack.org/#/c/136869/
>
> I also want to point out that this blueprint targets all the drivers, not
> just Hyper-V. This blueprint targets all the users that desire to deploy
> instances with Windows guests and desire password-less authentication, the
> same way users can ssh into Linux-type guests.
>
> Best regards,
>
> Claudiu Belu
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Nova API meeting

[Christopher Yeoh]

Hi,

Just a reminder that the weekly Nova API meeting is being held tomorrow
Friday UTC . 

We encourage cloud operators and those who use the REST API such as
SDK developers and others who and are interested in the future of the
API to participate.

In other timezones the meeting is at:

EST 20:00 (Thu)
Japan 09:00 (Fri)
China 08:00 (Fri)
ACDT 10:30 (Fri)

The proposed agenda and meeting details are here: 

https://wiki.openstack.org/wiki/Meetings/NovaAPI

Please feel free to add items to the agenda. 

Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Cross-Repo Dependencies in Zuul

[Christopher Yeoh]

Wow, this is great! Thank you!

Chris

On Wed, Feb 11, 2015 at 8:56 AM, James E. Blair  wrote:

> Hi,
>
> We have added support for cross-repo dependencies (CRD) in Zuul.  The
> important bits:
>
> * To use them, include "Depends-On: " in the footer of
>   your commit message.  Use the full Change-ID ('I' + 40 characters).
>
> * These are one-way dependencies only -- do not create a cycle.
>
> * This is what all the grey dots and lines are in the check pipeline.
>
> Cross-Repo Dependencies Explained
> =
>
> There are two behaviors that might go by the name "cross-repo
> dependencies".  We call them one-way and multi-way.
>
> Multi-way CRD allow for bidirectional links between changes.  For
> instance, A depends on B and B depends on A.  The theory there is that
> both would be tested together and merged as a unit.  This is _not_ what
> we have implemented in Zuul.  Discussions over the past two years have
> revealed that this type of behavior could cause problems for continuous
> deploments as it means that two components must be upgraded
> simultaneously.  Not supporting this behavior is a choice we have made.
>
> One-way CRD behaves like a directed acyclic graph (DAG), like git
> itself, to indicate a one-way dependency relationship between changes in
> different git repos.  Change A may depend on B, but B may not depend on
> A.  This is what we have implemented in Zuul.
>
> Gate Pipeline
> =
>
> When Zuul sees CRD changes, it serializes them in the usual manner when
> enqueuing them into a pipeline.  This means that if change A depends on
> B, then when they are added to the gate pipeline, B will appear first
> and A will follow.  If tests for B fail, both B and A will be removed
> from the pipeline, and it will not be possible for A to merge until B
> does.
>
> Note that if changes with CRD do not share a change queue (such as the
> "integrated gate" then Zuul is unable to enqueue them together, and the
> first will be required to merge before the second is enqueued.
>
> Check Pipeline
> ==
>
> When changes are enqueued into the check pipeline, all of the related
> dependencies (both normal git-dependencies that come from parent commits
> as well as CRD changes) appear in a dependency graph, as in gate.  This
> means that even in the check pipeline, your change will be tested with
> its dependency.  So changes that were previously unable to be fully
> tested until a related change landed in a different repo may now be
> tested together from the start.
>
> All of the changes are still independent (so you will note that the
> whole pipeline does not share a graph as in gate), but for each change
> tested, all of its dependencies are visually connected to it, and they
> are used to construct the git references that Zuul uses when testing.
> When looking at this graph on the status page, you will note that the
> dependencies show up as grey dots, while the actual change tested shows
> up as red or green.  This is to indicate that the grey changes are only
> there to establish dependencies.  Even if one of the dependencies is
> also being tested, it will show up as a grey dot when used as a
> dependency, but separately and additionally will appear as its own red
> or green dot for its test.
>
> (If you don't see grey dots on the status page, reload the page to get
> the latest version.)
>
> Multiple Changes
> 
>
> A Gerrit change ID may refer to multiple changes (on multiple branches
> of the same project, or even multiple projects).  In these cases, Zuul
> will treat all of the changes with that change ID as dependencies.  So
> if you say that a tempest change Depends-On a change ID that has changes
> in nova master and nova stable/juno, then when testing the tempest
> change, both nova changes will be applied, and when deciding whether the
> tempest change can merge, both changes must merge ahead of it.
>
> A change may depend on more than one Gerrit change ID as well.  So it is
> possible for a change in tempest to depend on a change in devstack and a
> change in nova.  Simply add more "Depends-On:" lines to the footer.
>
> Cycles
> ==
>
> If a cycle is created by use of CRD, Zuul will abort its work very
> early.  There will be no message in Gerrit and no changes that are part
> of the cycle will be enqueued into any pipeline.  This is to protect
> Zuul from infinite loops.  I hope that we can improve this to at least
> leave a message in Gerrit in the future.  But in the meantime, please be
> cognizant of this and do not create dependency cycles with Depends-On
> lines.
>
> Examples
> 
>
> The following two infra changes have been tested together because of the
> Depends-On: line in the commit message of the first:
>
>   https://review.openstack.org/#/c/152508/
>   https://review.openstack.org/#/c/152504/
>
> In fact, you can see earlier test results failing until it was rechecked
> after CRD went into production (aroun

Re: [openstack-dev] [nova] will the real v2.1/v3 API status please stand up?

[Christopher Yeoh]

Hi,

On Sat, Feb 7, 2015 at 8:35 AM, Matt Riedemann 
wrote:

> I'm not going to hide it, I don't know what's going on with the v2.1 API
> status, i.e. what is the criteria to that thing dropping it's
> 'experimental' label?
>
>
So I caught up with Matt on IRC, repeating some references and discussion
here for everyone else


> I wasn't at the mid-cycle meetup for Kilo but even for Juno I'll admit I
> was a bit lost. It's not my fault, I'm more good looks than brains. :)
>
> When I look at approved specs for Kilo, three pop out:
>
> 1. https://blueprints.launchpad.net/nova/+spec/v2-on-v3-api
>
> 2. https://blueprints.launchpad.net/nova/+spec/api-microversions
>
> 3. https://blueprints.launchpad.net/nova/+spec/v3-api-policy
>
>
So we need the first to blueprints for v2.1 microversions. We don't need
v3-api-policy merged to release
 v2.1 microversions though I believe it is a separate important bit of work
to reduce tech debt and make
life easier for operators.




> The only one of those that has a dependency in launchpad is the last one
> and it's dependency is on:
>
> https://blueprints.launchpad.net/nova/+spec/nova-v3-api
>
> Which looks like it was replaced by the v2-on-v3-api blueprint.
>
> If I look at the open changes for each, there are a lot:
>
> 1. https://review.openstack.org/#/q/status:open+project:
> openstack/nova+branch:master+topic:bp/v2-on-v3-api,n,z
>
> 2. https://review.openstack.org/#/q/status:open+project:
> openstack/nova+branch:master+topic:bp/api-microversions,n,z
>
> 3. https://review.openstack.org/#/q/status:open+project:
> openstack/nova+branch:master+topic:bp/v3-api-policy,n,z
>
> Do those all need to merge before the v2.1 API is no longer experimental?
>
>
We have an etherpad here which tracks our release criteria for v2.1 and
microversions:

https://etherpad.openstack.org/p/v2_1_ReleaseCriteria

As mentioned above it doesn't include api-policy

To make life easier for us I'd also like to request that if you review a
changeset that modifies the v2 api that you ensure it also if required is
applied to v2.1(v3 code). If it doesn't apply to v3 then ensure a
v2-only tag is in the commit message. That will help us verify v2 does not
diverge from v2.1 just before
release. After that I think v2 code will be essentially frozen except for
bug fixes and any api changes will
only be made through microversions.

Regards,

Chris

Is the, for lack of a better term, 'completion criteria', being tracked in
> an etherpad or wiki page somewhere?  I see stuff in the priorities etherpad
> https://etherpad.openstack.org/p/kilo-nova-priorities-tracking but it's
> not clear to me at a high level what makes v2.1 no longer experimental.
>
> Can someone provide that in less than 500 words?
>
> --
>
> Thanks,
>
> Matt Riedemann
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python-novaclient][nova] future of --os-compute-api-version option and whole api versioning

[Christopher Yeoh]

On Thu, Feb 5, 2015 at 11:37 AM, Andrey Kurilin 
wrote:

> First results is https://review.openstack.org/#/c/152569/
>
> > - if os-compute-api-version is not supplied don't send any header at all
> > - it is probably worth doing a bit version parsing to see if it makes
> sense eg of format:
> >  r"^([1-9]\d*)\.([1-9]\d*|0)$" or latest
>
> implemented
>
>
> >- handle  HTTPNotAcceptable if the user asked for a version which is not
> supported
> >  (can also get a badrequest if its badly formatted and got through the
> novaclient filter)
> Based on https://review.openstack.org/#/c/150641/ , HTTPNotFound (404)
> will be returned by API, so the current implementation of novaclient is not
> required any changes.
>
>
So a microversion (say v2.12) is a concept which covers the API as a whole,
not just an extension. And as a result there is a concept of a minimum
version supported (currently 2.1) and a maximum). If a client requests a
version which it outside of min<->max they will have a HTTPNotAcceptable
returned.

If a client a requests a version which is valid, but the actual resource
point they attempt to access does not support an implementation (say the
version requested did not support anything yet), they will get a 404 (eg it
pretends not to be there).



> > - show the version header information returned
> Imo, API should return exception with proper message which already include
> this information.
>

Yep, if you request a version which doesn't fit within the global supported
(min<->max) you will get  a 406 which specifies min/max api versions for
that server.

For any other request you get returned an X-OpenStack-Compute-API-Version
header which
specifies the version used - which you may nto have exactly known on the
request if you didn't specify a version (eg not specified or 'latest')

Regards,

Chris


>
> On Mon, Feb 2, 2015 at 2:02 PM, Andrey Kurilin 
> wrote:
>
>> Thanks for the summary, I'll try to send first patch(maybe WIP) in few
>> days.
>>
>> On Mon, Feb 2, 2015 at 1:43 PM, Christopher Yeoh 
>> wrote:
>>
>>>
>>>
>>> On Sat, Jan 31, 2015 at 4:09 AM, Andrey Kurilin 
>>> wrote:
>>>
>>>> Thanks for the answer. Can I help with implementation of novaclient
>>>> part?
>>>>
>>>
>>> Sure! Do you think its something you can get proposed into Gerrit by the
>>> end of the week or very soon after?
>>> Its the sort of timeframe we're looking for to get microversions enabled
>>> asap I guess just let me know if it
>>> turns out you don't have the time.
>>>
>>> So I think a short summary of what is needed is:
>>> - if os-compute-api-version is not supplied don't send any header at all
>>> - it is probably worth doing a bit version parsing to see if it makes
>>> sense eg of format:
>>>  r"^([1-9]\d*)\.([1-9]\d*|0)$" or latest
>>> - handle  HTTPNotAcceptable if the user asked for a version which is not
>>> supported
>>>   (can also get a badrequest if its badly formatted and got through the
>>> novaclient filter)
>>> - show the version header information returned
>>>
>>> Regards,
>>>
>>> Chris
>>>
>>>
>>>> On Wed, Jan 28, 2015 at 11:50 AM, Christopher Yeoh 
>>>> wrote:
>>>>
>>>>> On Fri, 23 Jan 2015 15:51:54 +0200
>>>>> Andrey Kurilin  wrote:
>>>>>
>>>>> > Hi everyone!
>>>>> > After removing nova V3 API from novaclient[1], implementation of v1.1
>>>>> > client is used for v1.1, v2 and v3 [2].
>>>>> > Since we moving to micro versions, I wonder, do we need such
>>>>> > mechanism of choosing api version(os-compute-api-version) or we can
>>>>> > simply remove it, like in proposed change - [3]?
>>>>> > If we remove it, how micro version should be selected?
>>>>> >
>>>>>
>>>>> So since v3 was never officially released I think we can re-use
>>>>> os-compute-api-version for microversions which will map to the
>>>>> X-OpenStack-Compute-API-Version header. See here for details on what
>>>>> the header will look like. We need to also modify novaclient to handle
>>>>> errors when a version requested is not supported by the server.
>>>>>
>>>>> If the user does not specify a version number then we should not send
>>>>> anything at all. The serve

Re: [openstack-dev] [api][nova] Openstack HTTP error codes

[Christopher Yeoh]

On Tue, Feb 3, 2015 at 9:05 AM, Jay Pipes  wrote:

> On 01/29/2015 12:41 PM, Sean Dague wrote:
>
>> Correct. This actually came up at the Nova mid cycle in a side
>> conversation with Ironic and Neutron folks.
>>
>> HTTP error codes are not sufficiently granular to describe what happens
>> when a REST service goes wrong, especially if it goes wrong in a way
>> that would let the client do something other than blindly try the same
>> request, or fail.
>>
>> Having a standard json error payload would be really nice.
>>
>> {
>>   fault: ComputeFeatureUnsupportedOnInstanceType,
>>   messsage: "This compute feature is not supported on this kind of
>> instance type. If you need this feature please use a different instance
>> type. See your cloud provider for options."
>> }
>>
>> That would let us surface more specific errors.
>>
> 
>
>>
>> Standardization here from the API WG would be really great.
>>
>
> What about having a separate HTTP header that indicates the "OpenStack
> Error Code", along with a generated URI for finding more information about
> the error?
>
> Something like:
>
> X-OpenStack-Error-Code: 1234
> X-OpenStack-Error-Help-URI: http://errors.openstack.org/1234
>
> That way is completely backwards compatible (since we wouldn't be changing
> response payloads) and we could handle i18n entirely via the HTTP help
> service running on errors.openstack.org.
>
>
So I'm +1 to adding the X-OpenStack-Error-Code header assuming the error
code is unique
across OpenStack APIs and it has a fixed meaning (we never change it,
create a new one if
a project has a need for an error code which is close to the original one
but a bit different)

The X-OpenStack-Error-Help-URI header I'm not sure about. We can't
guarantee that apps will have
access to errors.openstack.org - is there an assumption here that we'd
build/ship an error translation service?

Regards,

Chris
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python-novaclient][nova] future of --os-compute-api-version option and whole api versioning

[Christopher Yeoh]

On Sat, Jan 31, 2015 at 4:09 AM, Andrey Kurilin 
wrote:

> Thanks for the answer. Can I help with implementation of novaclient part?
>

Sure! Do you think its something you can get proposed into Gerrit by the
end of the week or very soon after?
Its the sort of timeframe we're looking for to get microversions enabled
asap I guess just let me know if it
turns out you don't have the time.

So I think a short summary of what is needed is:
- if os-compute-api-version is not supplied don't send any header at all
- it is probably worth doing a bit version parsing to see if it makes sense
eg of format:
 r"^([1-9]\d*)\.([1-9]\d*|0)$" or latest
- handle  HTTPNotAcceptable if the user asked for a version which is not
supported
  (can also get a badrequest if its badly formatted and got through the
novaclient filter)
- show the version header information returned

Regards,

Chris


> On Wed, Jan 28, 2015 at 11:50 AM, Christopher Yeoh 
> wrote:
>
>> On Fri, 23 Jan 2015 15:51:54 +0200
>> Andrey Kurilin  wrote:
>>
>> > Hi everyone!
>> > After removing nova V3 API from novaclient[1], implementation of v1.1
>> > client is used for v1.1, v2 and v3 [2].
>> > Since we moving to micro versions, I wonder, do we need such
>> > mechanism of choosing api version(os-compute-api-version) or we can
>> > simply remove it, like in proposed change - [3]?
>> > If we remove it, how micro version should be selected?
>> >
>>
>> So since v3 was never officially released I think we can re-use
>> os-compute-api-version for microversions which will map to the
>> X-OpenStack-Compute-API-Version header. See here for details on what
>> the header will look like. We need to also modify novaclient to handle
>> errors when a version requested is not supported by the server.
>>
>> If the user does not specify a version number then we should not send
>> anything at all. The server will run the default behaviour which for
>> quite a while will just be v2.1 (functionally equivalent to v.2)
>>
>>
>> http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/api-microversions.html
>>
>>
>> >
>> > [1] - https://review.openstack.org/#/c/138694
>> > [2] -
>> >
>> https://github.com/openstack/python-novaclient/blob/master/novaclient/client.py#L763-L769
>> > [3] - https://review.openstack.org/#/c/149006
>> >
>>
>>
>
>
> --
> Best regards,
> Andrey Kurilin.
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Product] [all][log] Openstack HTTP error codes

[Christopher Yeoh]

On Sun, Feb 1, 2015 at 2:57 AM, Sean Dague  wrote:

> On 01/31/2015 05:24 AM, Duncan Thomas wrote:
> > Hi
> >
> > This discussion came up at the cinder mid-cycle last week too,
> > specifically in the context of 'Can we change the details text in an
> > existing error, or is that an unacceptable API change'.
> >
> > I have to second security / operational concerns about exposing too much
> > granularity of failure in these error codes.
> >
> > For cases where there is something wrong with the request (item out of
> > range, invalid names, feature not supported, etc) I totally agree that
> > we should have good, clear, parsable response, and standardisation would
> > be good. Having some fixed part of the response (whether a numeric code
> > or, as I tend to prefer, a CamelCaseDescription so that I don't have to
> > go look it up) and a human readable description section that is subject
> > to change seems sensible.
> >
> > What I would rather not see is leakage of information when something
> > internal to the cloud goes wrong, that the tenant can do nothing
> > against. We certainly shouldn't be leaking internal implementation
> > details like vendor details - that is what request IDs and logs are for.
> > The whole point of the cloud, to me, is that separation between the
> > things a tenant controls (what they want done) and what the cloud
> > provider controls (the details of how the work is done).
> >
> > For example, if a create volume request fails because cinder-scheduler
> > has crashed, all the tenant should get back is 'Things are broken, try
> > again later or pass request id 1234-5678-abcd-def0 to the cloud admin'.
> > They should need to or even be allowed to care about the details of the
> > failure, it is not their domain.
>
> Sure, the value really is in determining things that are under the
> client's control to do differently. A concrete one is a multi hypervisor
> cloud with 2 hypervisors (say kvm and docker). The volume attach
> operation to a docker instance (which presumably is a separate set of
> instance types) can't work. The user should be told that that can't work
> with this instance_type if they try it.
>
> That's actually user correctable information. And doesn't require a
> ticket to move forward.
>
> I also think we could have a detail level knob, because I expect the
> level of information exposure might be considered different in public
> cloud use case vs. a private cloud at an org level or a private cloud at
> a dept level.
>
>
That could turn into a major compatibility issue if what we returned could
(and
probably would between public/private) change between clouds? If we want to
encourage
people to parse this sort of thing I think we need to settle on whether we
send the
information back or not for everyone.


> -Sean
>
> >
> >
> >
> > On 30 January 2015 at 02:34, Rochelle Grober  > > wrote:
> >
> > Hi folks!
> >
> > Changed the tags a bit because this is a discussion for all projects
> > and dovetails with logging rationalization/standards/
> >
> > At the Paris summit, we had a number of session on logging that kept
> > circling back to Error Codes.  But, these codes would not be http
> > codes, rather, as others have pointed out, codes related to the
> > calling entities and referring entities and the actions that
> > happened or didn’t.  Format suggestions were gathered from the
> > Operators and from some senior developers.  The Logging Working
> > Group is planning to put forth a spec for discussion on formats and
> > standards before the Ops mid-cycle meetup.
> >
> > Working from a Glance proposal on error codes:
> > https://review.openstack.org/#/c/127482/ and discussions with
> > operators and devs, we have a strawman to propose.  We also have a
> > number of requirements from Ops and some Devs.
> >
> > Here is the basic idea:
> >
> > Code for logs would have four segments:
> > Project Vendor/Component  Error
> > Catalog number Criticality
> > Def [A-Z] [A-Z] [A-Z]   -
> > [{0-9}|{A-Z}][A-Z] - [-]-   [0-9]
> > Ex.  CIN-   NA-
> >   0001-
> >2
> > Cinder   NetApp
> >   driver error no
> >   Criticality
> > Ex.  GLA-  0A-
> >0051
> >  3
> > Glance  Api
> >error no
> >  Criticality
> > Three letters for project,  Either a two letter vendor code or a
> > number and letter for 0+letter for internal component of project
> > (like API=0A, Controller =0C, etc),  four d

Re: [openstack-dev] [nova] novaclient support for V2.1 micro versions

[Christopher Yeoh]

On Fri, 23 Jan 2015 16:53:55 +
"Day, Phil"  wrote:

> Hi Folks,
> 
> Is there any support yet in novaclient for requesting a specific
> microversion ?   (looking at the final leg of extending
> clean-shutdown to the API, and wondering how to test this in devstack
> via the novaclient)
> 

No, sorry, something should up within a week.

Regards,

Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python-novaclient][nova] future of --os-compute-api-version option and whole api versioning

[Christopher Yeoh]

On Fri, 23 Jan 2015 15:51:54 +0200
Andrey Kurilin  wrote:

> Hi everyone!
> After removing nova V3 API from novaclient[1], implementation of v1.1
> client is used for v1.1, v2 and v3 [2].
> Since we moving to micro versions, I wonder, do we need such
> mechanism of choosing api version(os-compute-api-version) or we can
> simply remove it, like in proposed change - [3]?
> If we remove it, how micro version should be selected?
> 

So since v3 was never officially released I think we can re-use
os-compute-api-version for microversions which will map to the
X-OpenStack-Compute-API-Version header. See here for details on what
the header will look like. We need to also modify novaclient to handle
errors when a version requested is not supported by the server.

If the user does not specify a version number then we should not send
anything at all. The server will run the default behaviour which for
quite a while will just be v2.1 (functionally equivalent to v.2)

http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/api-microversions.html


> 
> [1] - https://review.openstack.org/#/c/138694
> [2] -
> https://github.com/openstack/python-novaclient/blob/master/novaclient/client.py#L763-L769
> [3] - https://review.openstack.org/#/c/149006
> 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Questions on pep8 F811 hacking check for microversion

[Christopher Yeoh]

On Tue, 06 Jan 2015 07:31:19 -0500
Jay Pipes  wrote:

> On 01/06/2015 06:25 AM, Chen CH Ji wrote:
> > Based on nova-specs api-microversions.rst
> > we support following function definition format, but it violate the
> > hacking rule pep8 F811 because duplicate function definition
> > we should use #noqa for them , but considering microversion may
> > live for long time ,
> > keep adding #noqa may be a little bit ugly, can anyone suggest a
> > good solution for it ? thanks
> >
> >  >   @api_version(min_version='2.1')
> >  >   def _version_specific_func(self, req, arg1):
> >  >  pass
> >  >
> >  >   @api_version(min_version='2.5')
> >  >   def _version_specific_func(self, req, arg1):
> >  >  pass
> 
> Hey Kevin,
> 
> This was actually one of my reservations about the proposed 
> microversioning implementation -- i.e. having functions that are
> named exactly the same, only decorated with the microversioning
> notation. It kinda reminds me of the hell of debugging C++ code that
> uses STL: how does one easily know which method one is in when inside
> a debugger?
> 
> That said, the only other technique we could try to use would be to
> not use a decorator and instead have a top-level dispatch function
> that would inspect the API microversion (only when the API version
> makes a difference to the output or input of that function) and then
> dispatch the call to a helper method that had the version in its name.
> 
> So, for instance, let's say you are calling the controller's GET 
> /$tenant/os-hosts method, which happens to get routed to the 
> nova.api.openstack.compute.contrib.hosts.HostController.index()
> method. If you wanted to modify the result of that method and the API 
> microversion is at 2.5, you might do something like:
> 
>   def index(self, req):
>   req_api_ver = utils.get_max_requested_api_version(req)
>   if req_api_ver == (2, 5):
>   return self.index_2_5(req)
>   return self.index_2_1(req)
> 
>   def index_2_5(self, req):
>   results = self.index_2_1(req)
>   # Replaces 'host' with 'host_name'
>   for result in results:
>   result['host_name'] = result['host']
>   del result['host']
>   return results
> 
>   def index_2_1(self, req):
>   # Would be a rename of the existing index() method on
>   # the controller
> 

So having to manually add switching code everything we have an API
patch I think is not only longer and more complicated but more error
prone when updating. If we change something at the core in the future it
means changing all the microversioned code rather than just the
switching architecture at the core of wsgi.


> Another option would be to use something like JSON-patch to determine 
> the difference between two output schemas and automatically translate 
> one to another... but that would be a huge effort.
> 
> That's the only other way I can think of besides disabling F811,
> which I really would not recommend, since it's a valuable safeguard
> against duplicate function names (especially duplicated test methods).

So I don't think we need to disable F811 in general - why not just
disable it for any method with the api_version decorator? On those ones
we can do checks on what is passed to api_version which will help
verify that there hasn't been a typo to an api_version decorator.

Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] [nova] [glance] Consistency in client side sorting

[Christopher Yeoh]

On Mon, 05 Jan 2015 11:10:41 -0500
Jay Pipes  wrote:

> >
> > Thoughts on getting consistency across all 3 projects (and possibly
> > others)?
> 
> Yeah, I personally like the second option as well, but agree that 
> consistency is the key (pun intended) here.
> 
> I would say let's make a decision on the standard to go with
> (possibly via the API or SDK working groups?) and then move forward
> with support for that option in all three clients (and continue to
> support the old behaviour for 2 release cycles, with deprecation
> markings as appropriate).

+1 to making this available in a consistent way. We need need to support
the old client behaviour for at least a couple of cycles (maybe a bit
longer) and the burden of doing so is pretty low. I don't think however
that we can drop the REST API behaviour that quickly. 2 cycles for API
deprecation in the past has been considered an insufficent length of
time because of app breakage

Regards,

Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Nominating Melanie Witt for python-novaclient-core

[Christopher Yeoh]

On Wed, Jan 28, 2015 at 9:11 AM, Michael Still  wrote:

> Greetings,
>
> I would like to nominate Melanie Witt for the python-novaclient-core team.
>
> (What is python-novaclient-core? Its a new group which will contain
> all of nova-core as well as anyone else we think should have core
> reviewer powers on just the python-novaclient code).
>
> Melanie has been involved with nova for a long time now. She does
> solid reviews in python-novaclient, and at least two current
> nova-cores have suggested her as ready for core review powers on that
> repository.
>
> Please respond with +1s or any concerns.
>
>
+1
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Complexity check and v2 API

[Christopher Yeoh]

Hi,

Given the timing (no spec approved) it sounds like a v2.1 plus
microversions (just merging) with no v2 changes at all.

The v2.1 framework is more flexible and you should need no changes to
servers.py at all as there are hooks for adding extra parameters in
separate plugins. There are examples of this in the v3 directory which is
really v2.1 now.

Chris
On Thu, 18 Dec 2014 at 3:49 am, Pasquale Porreca <
pasquale.porr...@dektech.com.au> wrote:

> Thank you for the answer.
>
> my API proposal won't be merged in kilo release since the deadline for
> approval is tomorrow, so I may propose the fix to lower the complexity
> in another way, what do you think about a bug fix?
>
> On 12/17/14 18:05, Matthew Gilliard wrote:
> > Hello Pasquale
> >
> >   The problem is that you are trying to add a new if/else branch into
> > a method which is already ~250 lines long, and has the highest
> > complexity of any function in the nova codebase. I assume that you
> > didn't contribute much to that complexity, but we've recently added a
> > limit to stop it getting any worse. So, regarding your 4 suggestions:
> >
> > 1/ As I understand it, v2.1 should be the same as v2 at the
> > moment, so they need to be kept the same
> > 2/ You can't ignore it - it will fail CI
> > 3/ No thank you. This limit should only ever be lowered :-)
> > 4/ This is 'the right way'. Your suggestion for the refactor does
> > sound good.
> >
> > I suggest a single patch that refactors and lowers the limit in
> > tox.ini.  Once you've done that then you can add the new parameter in
> > a following patch. Please feel free to add me to any patches you
> > create.
> >
> > Matthew
> >
> >
> >
> > On Wed, Dec 17, 2014 at 4:18 PM, Pasquale Porreca
> >  wrote:
> >> Hello
> >>
> >> I am working on an API extension that adds a parameter on create server
> >> call; to implement the v2 API I added few lines of code to
> >> nova/api/openstack/compute/servers.py
> >>
> >> In particular just adding something like
> >>
> >> new_param = None
> >> if self.ext_mgr.is_loaded('os-new-param'):
> >> new_param = server_dict.get('new_param')
> >>
> >> leads to a pep8 fail with message 'Controller.create' is too complex
> (47)
> >> (Note that in tox.ini the max complexity is fixed to 47 and there is a
> note
> >> specifying 46 is the max complexity present at the moment).
> >>
> >> It is quite easy to make this test pass creating a new method just to
> >> execute these lines of code, anyway all other extensions are handled in
> that
> >> way and one of most important stylish rule states to be consistent with
> >> surrounding code, so I don't think a separate function is the way to go
> >> (unless it implies a change in how all other extensions are handled
> too).
> >>
> >> My thoughts on this situation:
> >>
> >> 1) New extensions should not consider v2 but only v2.1, so that file
> should
> >> not be touched
> >> 2) Ignore this error and go on: if and when the extension will be
> merged the
> >> complexity in tox.ini will be changed too
> >> 3) The complexity in tox.ini should be raised to allow new v2 extensions
> >> 4) The code of that module should be refactored to lower the complexity
> >> (i.e. move the load of each extension in a separate function)
> >>
> >> I would like to know if any of my point is close to the correct
> solution.
> >>
> >> --
> >> Pasquale Porreca
> >>
> >> DEK Technologies
> >> Via dei Castelli Romani, 22
> >> 00040 Pomezia (Roma)
> >>
> >> Mobile +39 3394823805
> >> Skype paskporr
> >>
> >>
> >> ___
> >> OpenStack-dev mailing list
> >> OpenStack-dev@lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> --
> Pasquale Porreca
>
> DEK Technologies
> Via dei Castelli Romani, 22
> 00040 Pomezia (Roma)
>
> Mobile +39 3394823805
> Skype paskporr
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] UniqueConstraint for name and tenant_id in security group

[Christopher Yeoh]

So I think this is something we really should get agreement on across the
open stack API first before flipping back and forth on a case by case
basis.

Personally I think we should be using uuids for uniqueness and leave any
extra restrictions to a ui layer if really required. If we try to have name
uniqueness then "test " should be considered the same as " test" as " test
" and it introduces all sorts of slightly different combos that look the
same except under very close comparison. Add unicode for extra fun.

Chris
On Tue, 16 Dec 2014 at 7:24 am, Maru Newby  wrote:

>
> On Dec 15, 2014, at 9:13 AM, Assaf Muller  wrote:
>
> >
> >
> > - Original Message -
> >> -BEGIN PGP SIGNED MESSAGE-
> >> Hash: SHA512
> >>
> >> I was (rightfully) asked to share my comments on the matter that I
> >> left in gerrit here. See below.
> >>
> >> On 12/12/14 22:40, Sean Dague wrote:
> >>> On 12/12/2014 01:05 PM, Maru Newby wrote:
> 
>  On Dec 11, 2014, at 2:27 PM, Sean Dague  wrote:
> 
> > On 12/11/2014 04:16 PM, Jay Pipes wrote:
> >> On 12/11/2014 04:07 PM, Vishvananda Ishaya wrote:
> >>> On Dec 11, 2014, at 1:04 PM, Jay Pipes 
> >>> wrote:
>  On 12/11/2014 04:01 PM, Vishvananda Ishaya wrote:
> >
> > On Dec 11, 2014, at 8:00 AM, Henry Gessau
> >  wrote:
> >
> >> On Thu, Dec 11, 2014, Mark McClain 
> >> wrote:
> >>>
>  On Dec 11, 2014, at 8:43 AM, Jay Pipes
>  mailto:jaypi...@gmail.com>>
>  wrote:
> 
>  I'm generally in favor of making name attributes
>  opaque, utf-8 strings that are entirely
>  user-defined and have no constraints on them. I
>  consider the name to be just a tag that the user
>  places on some resource. It is the resource's ID
>  that is unique.
> 
>  I do realize that Nova takes a different approach
>  to *some* resources, including the security group
>  name.
> 
>  End of the day, it's probably just a personal
>  preference whether names should be unique to a
>  tenant/user or not.
> 
>  Maru had asked me my opinion on whether names
>  should be unique and I answered my personal
>  opinion that no, they should not be, and if
>  Neutron needed to ensure that there was one and
>  only one default security group for a tenant,
>  that a way to accomplish such a thing in a
>  race-free way, without use of SELECT FOR UPDATE,
>  was to use the approach I put into the pastebin
>  on the review above.
> 
> >>>
> >>> I agree with Jay.  We should not care about how a
> >>> user names the resource. There other ways to
> >>> prevent this race and Jay’s suggestion is a good
> >>> one.
> >>
> >> However we should open a bug against Horizon because
> >> the user experience there is terrible with duplicate
> >> security group names.
> >
> > The reason security group names are unique is that the
> > ec2 api supports source rule specifications by
> > tenant_id (user_id in amazon) and name, so not
> > enforcing uniqueness means that invocation in the ec2
> > api will either fail or be non-deterministic in some
> > way.
> 
>  So we should couple our API evolution to EC2 API then?
> 
>  -jay
> >>>
> >>> No I was just pointing out the historical reason for
> >>> uniqueness, and hopefully encouraging someone to find the
> >>> best behavior for the ec2 api if we are going to keep the
> >>> incompatibility there. Also I personally feel the ux is
> >>> better with unique names, but it is only a slight
> >>> preference.
> >>
> >> Sorry for snapping, you made a fair point.
> >
> > Yeh, honestly, I agree with Vish. I do feel that the UX of
> > that constraint is useful. Otherwise you get into having to
> > show people UUIDs in a lot more places. While those are good
> > for consistency, they are kind of terrible to show to people.
> 
>  While there is a good case for the UX of unique names - it also
>  makes orchestration via tools like puppet a heck of a lot simpler
>  - the fact is that most OpenStack resources do not require unique
>  names.  That being the case, why would we want security groups to
>  deviate from this convention?
> >>>
> >>> Maybe the other ones are the broken ones?
> >>>
> >>> Honestly, any sanely usable system makes names unique inside a
> >>> container. Like files in a directory.
> >>
> >> Correct. Or take git: it does not use hashes to identify objects, right?
> >>
> >>> In this case the tenant is the container, which makes se

Re: [openstack-dev] [qa] How to delete a VM which is in ERROR state?

[Christopher Yeoh]

If force delete doesn't work please do submit the bug report along with as
much of the relevant nova logs as you can. Even better if it's easily
repeatable with devstack.

Chris
On Sat, 13 Dec 2014 at 8:43 am, pcrews  wrote:

> On 12/09/2014 03:54 PM, Ken'ichi Ohmichi wrote:
> > Hi,
> >
> > This case is always tested by Tempest on the gate.
> >
> > https://github.com/openstack/tempest/blob/master/tempest/
> api/compute/servers/test_delete_server.py#L152
> >
> > So I guess this problem wouldn't happen on the latest version at least.
> >
> > Thanks
> > Ken'ichi Ohmichi
> >
> > ---
> >
> > 2014-12-10 6:32 GMT+09:00 Joe Gordon :
> >>
> >>
> >> On Sat, Dec 6, 2014 at 5:08 PM, Danny Choi (dannchoi) <
> dannc...@cisco.com>
> >> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I have a VM which is in ERROR state.
> >>>
> >>>
> >>> +--+
> --+++---
> --++
> >>>
> >>> | ID   | Name
> >>> | Status | Task State | Power State | Networks   |
> >>>
> >>>
> >>> +--+
> --+++---
> --++
> >>>
> >>> | 1cb5bf96-619c-4174-baae-dd0d8c3d40c5 |
> >>> cirros--1cb5bf96-619c-4174-baae-dd0d8c3d40c5 | ERROR  | -  |
> NOSTATE
> >>> ||
> >>>
> >>>
> >>> I tried in both CLI “nova delete” and Horizon “terminate instance”.
> >>> Both accepted the delete command without any error.
> >>> However, the VM never got deleted.
> >>>
> >>> Is there a way to remove the VM?
> >>
> >>
> >> What version of nova are you using? This is definitely a serious bug,
> you
> >> should be able to delete an instance in error state. Can you file a bug
> that
> >> includes steps on how to reproduce the bug along with all relevant logs.
> >>
> >> bugs.launchpad.net/nova
> >>
> >>>
> >>>
> >>> Thanks,
> >>> Danny
> >>>
> >>> ___
> >>> OpenStack-dev mailing list
> >>> OpenStack-dev@lists.openstack.org
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>
> >>
> >> ___
> >> OpenStack-dev mailing list
> >> OpenStack-dev@lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> Hi,
>
> I've encountered this in my own testing and have found that it appears
> to be tied to libvirt.
>
> When I hit this, reset-state as the admin user reports success (and
> state is set), *but* things aren't really working as advertised and
> subsequent attempts to do anything with the errant vm's will send them
> right back into 'FLAIL' / can't delete / endless DELETING mode.
>
> restarting libvirt-bin on my machine fixes this - after restart, the
> deleting vm's are properly wiped without any further user input to
> nova/horizon and all seems right in the world.
>
> using:
> devstack
> ubuntu 14.04
> libvirtd (libvirt) 1.2.2
>
> triggered via:
> lots of random create/reboot/resize/delete requests of varying validity
> and sanity.
>
> Am in the process of cleaning up my test code so as not to hurt anyone's
> brain with the ugly and will file a bug once done, but thought this
> worth sharing.
>
> Thanks,
> Patrick
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python-novaclient] Status of novaclient V3

[Christopher Yeoh]

> On 3 Dec 2014, at 10:00 pm, Joe Gordon  wrote:
> 
> 
> 
>> On Tue, Dec 2, 2014 at 5:21 PM, Andrey Kurilin  wrote:
>> Hi!
>> 
>> While working on fixing wrong import in novaclient v3 shell, I have found 
>> that a lot of commands, which are listed in V3 shell(novaclient.v3.shell), 
>> are broken, because appropriate managers are missed from V3 
>> client(novaclient.V3.client.Client).
>> 
>> Template of error is "ERROR (AttributeError): 'Client' object has no 
>> attribute ''", where  can be "floating_ip_pools", "floating_ip", 
>> "security_groups", "dns_entries" and etc.
>> 
>> I know that novaclient V3 is not finished yet, and I guess it will be not 
>> finished. So the main question is:
>>  What we should do with implemented code of novaclient V3 ? Should it be 
>> ported to novaclient V2.1 or it can be removed?
> 
> I think it can be removed, as we are not going forward with the V3 API. But I 
> will defer to Christopher Yeoh/Ken’ichi Ohmichi for the details.


I think it can all just be removed now. We're going to need to enhance nova 
client to understand micro versions instead. But for now v2.1 should look just 
like v2

Chris

>  
>> 
>> 
>> -- 
>> Best regards,
>> Andrey Kurilin.
>> 
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA][Tempest] Proposing Ghanshyam Mann for Tempest Core

[Christopher Yeoh]

+1

Sent from my iPad

> On 22 Nov 2014, at 4:56 am, Matthew Treinish  wrote:
> 
> 
> Hi Everyone,
> 
> I'd like to propose we add Ghanshyam Mann (gmann) to the tempest core team. 
> Over
> the past couple of cycles Ghanshyam has been actively engaged in the Tempest
> community. Ghanshyam has had one of the highest review counts on Tempest for
> the past cycle, and he has consistently been providing reviews that have been
> of consistently high quality that show insight into both the project internals
> and it's future direction. I feel that Ghanshyam will make an excellent 
> addition
> to the core team.
> 
> As per the usual, if the current Tempest core team members would please vote 
> +1
> or -1(veto) to the nomination when you get a chance. We'll keep the polls open
> for 5 days or until everyone has voted.
> 
> Thanks,
> 
> Matt Treinish
> 
> References:
> 
> https://review.openstack.org/#/q/reviewer:%22Ghanshyam+Mann+%253Cghanshyam.mann%2540nectechnologies.in%253E%22,n,z
> 
> http://stackalytics.com/?user_id=ghanshyammann&metric=marks
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Nova API meeting

[Christopher Yeoh]

Hi,

Just a reminder that the weekly Nova API meeting is being held tomorrow
Friday UTC . 

We encourage cloud operators and those who use the REST API such as
SDK developers and others who and are interested in the future of the
API to participate.

In other timezones the meeting is at:

EST 20:00 (Thu)
Japan 09:00 (Fri)
China 08:00 (Fri)
ACDT 10:30 (Fri)

The proposed agenda and meeting details are here: 

https://wiki.openstack.org/wiki/Meetings/NovaAPI

Please feel free to add items to the agenda. 

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] Counting resources

[Christopher Yeoh]

On Thu, 20 Nov 2014 14:47:16 +0100
Salvatore Orlando  wrote:

> Aloha guardians of the API!
> 
> I haven recently* reviewed a spec for neutron [1] proposing a
> distinct URI for returning resource count on list operations.
> This proposal is for selected neutron resources, but I believe the
> topic is general enough to require a guideline for the API working
> group. Your advice is therefore extremely valuable.
> 
> In a nutshell the proposal is to retrieve resource count in the
> following way:
> GET ///count
> 
> In my limited experience with RESTful APIs, I've never encountered
> one that does counting in this way. This obviously does not mean it's
> a bad idea. I think it's not great from a usability perspective to
> require two distinct URIs to fetch the first page and then the total
> number of elements. I reckon the first response page for a list
> operation might include also the total count. For example:
> 
> {'resources': [{meh}, {meh}, {meh_again}],
>  'resource_count': 55
>  }
> 
> I am however completely open to consider other alternatives.
> What is your opinion on this matter?

FWIW there is a nova spec proposed for counting resources as
well (I think it might have been previously approved for Juno). 

https://review.openstack.org/#/c/134279/

I haven't compared the two, but I can't think of a reason we'd
need to be any different between projects here.

Regards,

Chris

> 
> Regards,
> Salvatore
> 
> 
> * it's been 10 days now
> 
> [1] https://review.openstack.org/#/c/102199/


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] API-WG meeting (note time change this week)

[Christopher Yeoh]

On Wed, 19 Nov 2014 19:34:44 +
Everett Toews  wrote:
> 
> 2. Do you know if there is a way to subscribe to only the API WG
> meeting from that calendar?

I haven't been able to find a way to do that. Fortunately for me most
of the openstack meetings end up being between 12am and 8am so it
doesn't actually clutter up my calendar view ;-)

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [api] API-WG meeting (note time change this week)

[Christopher Yeoh]

Hi,

We have moved to alternating times each week for the API WG meeting so
people from other timezones can attend. Since this is an odd week 
the meeting will be Thursday UTC 1600. Details here:

https://wiki.openstack.org/wiki/Meetings/API-WG

The google ical feed hasn't been updated yet, but thats not surprising
since the wiki page was only updated a few hours ago.

Regards,

Chris


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Do we need to add xml support for new API extensions on v2 API ?

[Christopher Yeoh]

On Wed, 19 Nov 2014 13:11:40 +0800
Chen CH Ji  wrote:

> 
> Hi
>  I saw we are removing v2 XML support proposed
> several days before
> 
>  For new api extensions, do we need to add it now and
> remove it later or only support JSON ? Thanks

I don't think any api additions to the api should include XML support.

Regards,

Chris

> 
> Best Regards!
> 
> Kevin (Chen) Ji 纪 晨
> 
> Engineer, zVM Development, CSTL
> Notes: Chen CH Ji/China/IBM@IBMCN   Internet: jiche...@cn.ibm.com
> Phone: +86-10-82454158
> Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian
> District, Beijing 100193, PRC

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] policy on old / virtually abandoned patches

[Christopher Yeoh]

On Tue, Nov 18, 2014 at 11:44 PM, Jay Pipes  wrote:

> On 11/18/2014 07:29 AM, Daniel P. Berrange wrote:
>
>> On Tue, Nov 18, 2014 at 07:06:59AM -0500, Sean Dague wrote:
>>
>>> Nova currently has 197 patches that have seen no activity in the last 4
>>> weeks (project:openstack/nova age:4weeks status:open).
>>>
>>> Of these
>>>   * 108 are currently Jenkins -1 (project:openstack/nova age:4weeks
>>> status:open label:Verified<=-1,jenkins)
>>>   * 60 are -2 by a core team member (project:openstack/nova age:4weeks
>>> status:open label:Code-Review<=-2)
>>>
>>> (note, those 2 groups sometimes overlap)
>>>
>>> Regardless, the fact that Nova currently has 792 open reviews, and 1/4
>>> of them seem dead, seems like a cleanup thing we could do.
>>>
>>> I'd like to propose that we implement our own auto abandon mechanism
>>> based on reviews that are either held by a -2, or Jenkins -1 after 4
>>> weeks time. I can write a quick script to abandon with a friendly
>>> message about why we are doing it, and to restore it if work is
>>> continuing.
>>>
>>
>> Yep, purging anything that's older than 4 weeks with negative karma
>> seems like a good idea.  It'll make it easier for us to identify those
>> patches which are still "maintained" and target them for review.
>>
>> That said, there's some edge cases - for example I've got some patches
>> up for review that have a -2 on them, becase we're waiting for blueprint
>> approval. IIRC, previously we would post a warning about pending auto-
>> abandon a week before, and thus give the author the chance to add a
>> comment to prevent auto-abandon taking place. It would be neccessary to
>> have this ability to deal with the case where we're just temporarily
>> blocked on other work.
>>
>
> Yes, this is indeed an issue. However, couldn't we just say "Add a
> -Workflow label to avoid auto-abandon" and then have the script simply
> ignore patches with -Workflow?


+1 for not auto abandoning patches marked WIP (or at least make the timeout
much longer for those)

Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] fix latency on requirements breakage

[Christopher Yeoh]

On Tue, Nov 18, 2014 at 9:32 AM, Sean Dague  wrote:

> waiting extra long for valid test results. People don't realize their
> code can't pass and just keep pushing patches up consuming resources
> which means that parts of the project that could pass tests, is backed
> up behind 100% guarunteed failing parts. All in all, not a great system.
>
>
Maybe a MOTD at the top of http://review.openstack.org could help here?
Have a button
that the QA/infra people can hit when everything is broken that puts up a
message
there asking people to stop rechecking/submitting patches.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] v2 or v3 for new api

[Christopher Yeoh]

On Tue, Nov 18, 2014 at 2:31 AM, Pasquale Porreca <
pasquale.porr...@dektech.com.au> wrote:

> Thank you very much Christopher
>
> On 11/17/14 12:15, Christopher Yeoh wrote:
>
>> Yes, sorry documentation has been on our todo list for too long. Could I
>> get you to submit a bug report about the lack of developer documentation
>> for api plugins? It might hurry us up :-)
>>
>
> I reported as a bug and subscribed you to it. https://bugs.launchpad.net/
> nova/+bug/1393455
>
>
Thanks!


>
>> In the meantime, off the top of my head. you'll need to create or
>> modify the following files in a typical plugin:
>>
>> setup.cfg - add an entry in at least the nova.api.v3.extensions section
>>
>> etc/nova/policy.json - an entry for the permissions for you plugin,
>> perhaps one per api method for maximum flexibility. Also will need a
>> discoverable entry (lots of examples in this file)
>>
>> nova/tests/unit/fake_policy.json (similar to policy.json)
>>
>
> I wish I had asked about this before, I found yet these files, but I
> confess it took quite a bit of time to guess I had to modify them (I
> actually didn't modify yet fake_policy, but my tests are still not
> completed).
> What about nova/nova.egg-info/entry_points.txt I mentioned earlier?
>
>
The entry points file is automatically updated based on setup.cfg


>
>> nova/api/openstack/compute/plugins/v3/ - please make the
>> alias name something os-scheduler-hints rather than OS-SCH-HNTS. No
>> skimping on vowels. Probably the easiest way at this stage without more
>> doco is look for for a plugin in that directory that does the sort of the
>> thing you want to do.
>>
>
> Following the path of other plugins, I created a module
> nova/api/openstack/compute/plugins/v3/node_uuid.py, while the class is
> NodeUuid(extensions.V3APIExtensionBase) the alias is os-node-uuid and the
> actual json parameter is node_uuid. I hope this is correct...
>
>
>> nova/tests/unit/nova/api/openstack/compute/contrib/test_your_plugin.py -
>> we have been combining the v2 and v2.1(v3) unittests to share as much as
>> possible, so please do the same here for new tests as the v3 directory will
>> be eventually removed. There's quite a few examples now in that directory
>> of sharing unittests between v2.1 and v2 but with a new extension the
>> customisation between the two should be pretty minimal (just a bit of
>> inheritance to call the right controller)
>>
>>
> Very good to know. I put my test in nova/tests/unit/api/openstack/plugins/v3
> , but I was getting confused by the fact only few tests were in this folder
> while the tests in nova/tests/unit/api/openstack/compute/contrib/ covered
> both v2 and v2.1 cases.
> So should I move my test in nova/tests/unit/api/openstack/compute/contrib/
> folder, right?
>

Yes


>
>  nova/tests/unit/integrated/v3/test_your_plugin.py
>> nova/tests/unit/integrated/test_api_samples.py
>>
>> Sorry the api samples tests are not unified yet. So you'll need to create
>> two. All of the v2 api sample tests are in one directory, whilst the the
>> v2.1 are separated into different files by plugin.
>>
>> There's some rather old documentation on how to generate the api samples
>> themselves (hint: directories aren't made automatically) here:
>>
>> https://blueprints.launchpad.net/nova/+spec/nova-api-samples
>>
>> Personally I wouldn't bother with any xml support if you do decide to
>> support v2 as its deprecated anyway.
>>
>
> After reading your answer I understood I have to work more on this part :)
>
>
>> Hope this helps. Feel free to add me as a reviewer for the api parts of
>> your changesets.
>>
>
> It helps a lot! I will add you for sure as soon as I will upload my code.
> For now the specification has still to be approved, so I think I have to
> wait before to upload it, is that correct?
>
> This is the blueprint link anyway: https://blueprints.launchpad.
> net/nova/+spec/use-uuid-v1
>
>
So it won't hurt to upload the code before the spec is approved if you're
looking for some early feedback, but I'd recommend setting it to Work in
Progress otherwise it's likely to get -2'd pending spec approval

Regards,

Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] v2 or v3 for new api

[Christopher Yeoh]

On Mon, Nov 17, 2014 at 7:30 PM, Pasquale Porreca <
pasquale.porr...@dektech.com.au> wrote:

>  Thank you for the clarification, yes I know about the
> blueprint/specification, I submitted yet them and the spec is currently
> under review :)
>
> I noticed there are several steps one has always to do to enable and make
> a v3 api to work and pass the tests. It would be awesome to have a
> guideline or something similar that explain these steps, but I didn't find
> anything in wiki or documentation.
>
>
Yes, sorry documentation has been on our todo list for too long. Could I
get you to submit a bug report about the lack of developer documentation
for api plugins? It might hurry us up :-)

In the meantime, off the top of my head. you'll need to create or
modify the following files in a typical plugin:

setup.cfg - add an entry in at least the nova.api.v3.extensions section

etc/nova/policy.json - an entry for the permissions for you plugin, perhaps
one per api method for maximum flexibility. Also will need a discoverable
entry (lots of examples in this file)

nova/tests/unit/fake_policy.json (similar to policy.json)

nova/api/openstack/compute/plugins/v3/ - please make the
alias name something os-scheduler-hints rather than OS-SCH-HNTS. No
skimping on vowels. Probably the easiest way at this stage without more
doco is look for for a plugin in that directory that does the sort of the
thing you want to do.

nova/tests/unit/nova/api/openstack/compute/contrib/test_your_plugin.py - we
have been combining the v2 and v2.1(v3) unittests to share as much as
possible, so please do the same here for new tests as the v3 directory will
be eventually removed. There's quite a few examples now in that directory
of sharing unittests between v2.1 and v2 but with a new extension the
customisation between the two should be pretty minimal (just a bit of
inheritance to call the right controller)

nova/tests/unit/integrated/v3/test_your_plugin.py
nova/tests/unit/integrated/test_api_samples.py

Sorry the api samples tests are not unified yet. So you'll need to create
two. All of the v2 api sample tests are in one directory, whilst the the
v2.1 are separated into different files by plugin.

There's some rather old documentation on how to generate the api samples
themselves (hint: directories aren't made automatically) here:

https://blueprints.launchpad.net/nova/+spec/nova-api-samples

Personally I wouldn't bother with any xml support if you do decide to
support v2 as its deprecated anyway.

Hope this helps. Feel free to add me as a reviewer for the api parts of
your changesets.

Regards,

Chris



> In particular I noticed I had to modify the file
> nova/nova.egg-info/entry_points.txt to make my v3 api to load, but this
> file seems not to be under versioning, is this file modified only after the
> changes are merged?
>
>
> On 11/16/14 23:55, Christopher Yeoh wrote:
>
>  On Thu, Nov 13, 2014 at 12:14 AM, Pasquale Porreca <
> pasquale.porr...@dektech.com.au> wrote:
>
>> Hello
>>
>> I am working on an api for a new feature in nova, but I am wondering what
>> is the correct way to add a new extension: should it be supported by v2, v3
>> or both?
>>
>>
>  You need now to have at least a v2.1 (formerly known as v3) extension.
> V2 support if you want but I think once v2.1 is fully merged and tested
> (which may not be that far away at all) we should freeze v2 and rely just
> on v2.1 for new features. Otherwise the interaction between v2.1 being
> exactly equivalent to v2 plus having microversion support for v2.1 will get
> a bit confusing.
>
> As the other Chris mentioned, the first step however is to get a nova-spec
> submitted which needs to fully describe the API additions that you want to
> make.
>
>  Regards,
>
>  Chris
>
>  BR
>>
>> --
>> Pasquale Porreca
>>
>> DEK Technologies
>> Via dei Castelli Romani, 22
>> 00040 Pomezia (Roma)
>>
>> Mobile +39 3394823805
>> Skype paskporr
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> ___
> OpenStack-dev mailing 
> listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> --
> Pasquale Porreca
>
> DEK Technologies
> Via dei Castelli Romani, 22
> 00040 Pomezia (Roma)
>
> Mobile +39 3394823805
> Skype paskporr
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] Cross-Project Liaison for the API Working Group

[Christopher Yeoh]

On Sat, Nov 15, 2014 at 7:26 AM, Everett Toews 
wrote:

> On Nov 14, 2014, at 1:43 PM, Jay Pipes  wrote:
>
> > On 11/14/2014 05:13 PM, Everett Toews wrote:
> >> The liaison should be a core reviewer for the project, but does not
> >> need to be the PTL. By default, the liaison will be the PTL.
> >
> > ]Anyway, the outcome of the email exchange with Eoghan was that I
> recommended he submit a core for the API liaison to start, and that I would
> raise the issue on the ML to see if those conditions may be loosened to
> include non-cores. And... here is that issue being raised :)
>
> I’m totally fine with that. Ideally it’s the person who is the most
> passionate about the API from the team, regardless of core status.
>
> The current wording on the Cross-Project Liaisons page is
>
> "The liaison should be a core reviewer for the project, but does not need
> to be the PTL.”
>
> I think the key word there is _should_. Naturally, it’s preferable to want
> a core reviewer in this role because they have more say about what gets
> into the code base but it’s not an absolute must.
>
> We can loosen the wording further but I think it’s okay to proceed with a
> non-core reviewer, especially if that person is passionate about APIs.
>
>
My 2c is we should say "The liason should be the PTL or whomever they
delegate to be their representative"  and not mention anything about the
person needing to be a core developer. It removes any ambiguity about who
ultimately decides who the liason is (the PTL) without saying that they
have to do the work themselves.

Regards,

Chris



> Everett
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] v2 or v3 for new api

[Christopher Yeoh]

On Thu, Nov 13, 2014 at 12:14 AM, Pasquale Porreca <
pasquale.porr...@dektech.com.au> wrote:

> Hello
>
> I am working on an api for a new feature in nova, but I am wondering what
> is the correct way to add a new extension: should it be supported by v2, v3
> or both?
>
>
You need now to have at least a v2.1 (formerly known as v3) extension. V2
support if you want but I think once v2.1 is fully merged and tested (which
may not be that far away at all) we should freeze v2 and rely just on v2.1
for new features. Otherwise the interaction between v2.1 being exactly
equivalent to v2 plus having microversion support for v2.1 will get a bit
confusing.

As the other Chris mentioned, the first step however is to get a nova-spec
submitted which needs to fully describe the API additions that you want to
make.

Regards,

Chris

BR
>
> --
> Pasquale Porreca
>
> DEK Technologies
> Via dei Castelli Romani, 22
> 00040 Pomezia (Roma)
>
> Mobile +39 3394823805
> Skype paskporr
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [api] API WG Meeting

[Christopher Yeoh]

Hi,

The patch setting the time for the API Working Group meeting has
merged: 

https://review.openstack.org/#/c/128332/

The time for the meeting is Thursday UTC  

In other timezones the meeting is at:

EST 20:00 (Wed)
Japan 09:00 (Thu)
China 08:00 (Thu)
ACDT 10:30 (Thu)

The meeting details are here: 

https://wiki.openstack.org/wiki/Meetings/API-WG

I know this is very short notice and some people have already started
travelling for Summit. But I'll be there in case anyone else turns up.
We won't of course have a meeting next week.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] API Workgroup git repository

[Christopher Yeoh]

On Thu, 23 Oct 2014 05:53:21 +
Shaunak Kashyap  wrote:

> Thanks Chris. And am I correct in assuming that no weekly api-wg
> meetings are currently going on? I don’t see anything here anyway:
> https://wiki.openstack.org/wiki/Meetings.
> 

They haven't started yet. Not sure if we'll be squeezing one in before
summit or not. The proposed meeting time patch only just recently
merged (Thursdays UTC )

https://review.openstack.org/#/c/128332/2

so perhaps thats why its not there yet.

Chris

> Shaunak
> 
> On Oct 22, 2014, at 10:39 PM, Christopher Yeoh
> mailto:cbky...@gmail.com>> wrote:
> 
> On Thu, Oct 23, 2014 at 3:51 PM, Shaunak Kashyap
> mailto:shaunak.kash...@rackspace.com>>
> wrote: If I have a proposal to make for a guideline, how do I do it?
> Do I simply create a gerrit review for a file under
> http://git.openstack.org/cgit/openstack/api-wg/tree/guidelines?
> 
> 
> Yes, either additions to the existing files or add new ones if there
> is nothing currently appropriate.
> 
> We still need to talk about the exact process for accepting a patch
> yet, but I was thinking about something along the lines of (and
> haven't really discussed it with anyone yet):
> 
> - must be up for at least a week
> - discussed at a weekly api-wg meeting
> - no more than one negative vote (ok we might need something to break
>   a deadlock if we really can't get consensus over something but
>   I think there should be a lot of incentive to try very hard
>   to do so).
> 
> Though for now any setup work in the repository is getting approved
> by Jay or I if it looks reasonable.
> 
> Chris
> 
> 
> Shaunak
> 
> On Oct 22, 2014, at 3:34 PM, Christopher Yeoh
> mailto:cbky...@gmail.com>> wrote:
> 
> > On Wed, 22 Oct 2014 20:36:27 +
> > Everett Toews
> > mailto:everett.to...@rackspace.com>>
> > wrote:
> >
> >> I notice at the top of the GitHub mirror page [1] it reads, "API
> >> Working Group http://openstack.org<http://openstack.org/>”
> >>
> >> Can we get that changed to "API Working Group
> >> https://wiki.openstack.org/wiki/API_Working_Group”?
> >>
> >> That URL would be much more helpful to people who come across the
> >> GitHub repo. It's not a code change so we would need a repo owner
> >> to actually make the change. Who should I contact about that?
> >
> > I think this will do it:
> >
> > https://review.openstack.org/130377
> >
> > Chris
> >
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] API Workgroup git repository

[Christopher Yeoh]

On Thu, Oct 23, 2014 at 3:51 PM, Shaunak Kashyap <
shaunak.kash...@rackspace.com> wrote:

> If I have a proposal to make for a guideline, how do I do it? Do I simply
> create a gerrit review for a file under
> http://git.openstack.org/cgit/openstack/api-wg/tree/guidelines?
>
>
Yes, either additions to the existing files or add new ones if there is
nothing currently appropriate.

We still need to talk about the exact process for accepting a patch yet,
but I was thinking about something along the lines of (and haven't really
discussed it with anyone yet):

- must be up for at least a week
- discussed at a weekly api-wg meeting
- no more than one negative vote (ok we might need something to break
  a deadlock if we really can't get consensus over something but
  I think there should be a lot of incentive to try very hard
  to do so).

Though for now any setup work in the repository is getting approved by Jay
or I if it looks reasonable.

Chris



> Shaunak
>
> On Oct 22, 2014, at 3:34 PM, Christopher Yeoh  wrote:
>
> > On Wed, 22 Oct 2014 20:36:27 +
> > Everett Toews  wrote:
> >
> >> I notice at the top of the GitHub mirror page [1] it reads, "API
> >> Working Group http://openstack.org”
> >>
> >> Can we get that changed to "API Working Group
> >> https://wiki.openstack.org/wiki/API_Working_Group”?
> >>
> >> That URL would be much more helpful to people who come across the
> >> GitHub repo. It's not a code change so we would need a repo owner to
> >> actually make the change. Who should I contact about that?
> >
> > I think this will do it:
> >
> > https://review.openstack.org/130377
> >
> > Chris
> >
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] API Workgroup git repository

[Christopher Yeoh]

On Wed, 22 Oct 2014 20:36:27 +
Everett Toews  wrote:

> I notice at the top of the GitHub mirror page [1] it reads, "API
> Working Group http://openstack.org”
> 
> Can we get that changed to "API Working Group
> https://wiki.openstack.org/wiki/API_Working_Group”?
> 
> That URL would be much more helpful to people who come across the
> GitHub repo. It's not a code change so we would need a repo owner to
> actually make the change. Who should I contact about that?

I think this will do it:

https://review.openstack.org/130377

Chris


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] API Workgroup git repository

[Christopher Yeoh]

On Wed, 22 Oct 2014 14:44:26 -0500
Anne Gentle  wrote:

> On Wed, Oct 22, 2014 at 2:26 PM, Steve Martinelli
>  wrote:
> 
> > we could set up a job to publish under docs.o.org/api-wg pretty
> > easily - it seems like a good place to start to publish this
> > content.
> >
> > thanks for getting the repo all setup chris and jay.
> >
> >
> Thanks for setting up the repo.
> 
> We probably want it to go to docs.openstack.org/developer/api-wg and
> link to it from here:
> 
> http://docs.openstack.org/developer/openstack-projects.html


That sounds good to me. Steve has a proposed patch we can use here:

https://review.openstack.org/#/c/130363/

I'm not sure of the syntax myself (I had a bit of a look into it last
night)

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [api] API Workgroup git repository

[Christopher Yeoh]

Hi,

The API Workgroup git repository has been setup and you can access it
here.

http://git.openstack.org/cgit/openstack/api-wg/

There is some content there though not all the proposed guidelines from
the wiki page are in yet:

https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines

Please feel free to start submitting patches to the document.

I have submitted a patch to convert the initial content from markdown to
rst and setup the tox targets to produce an html document. Seemed to be
an easier route as it seems to be the preferred format for OpenStack
projects and we can just copy all the build/check bits from the specs
repositories. Also doesn't require any changes to required packages.

https://review.openstack.org/130120

Until this is merged its probably better to base any patches on this
one.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] Request Validation - Stoplight

[Christopher Yeoh]

On Tue, Oct 21, 2014 at 12:15 PM, Kenichi Oomichi  wrote:

> Hi Amit,
>
> Thanks for picking this topic up,
>
> Honestly I don't have a strong opinion about validation libraries.
> Each project implements based on different web frameworks and
> the options of validation libraries would be limited from its web
> framework. For example, Nova implements its own wsgi framework and
> it is difficult to use pecan/wsme due to its API routing/parameter
> names. So Nova uses jsonschema for a new API(Nova v2.1 API) because
> of its flexibility and portability.
> From quick seeing, stoplight seems flexible and it could cover many
> cases. That would be nice for me, but I'm not sure that stoplight is
> the best library because jsonschema is a common way/library and portable.
>
> Related to this topic, I'd like to suggest that we have common validation
> patterns across OpenStack projects. Now each project contains its owns
> validation patterns for the other project's resource. For example, Nova
> contains validation patterns for project-id and image-id on the code[1].
> Ideally, these validation patterns would be nice to be ported/shared from
> Keystone and Glance and it is the best to use the same validation patterns
> between whole OpenStack projects for consistent interfaces. Maybe we can
> implement these patterns even if using different validation libraries.
>
>
This sounds good. Would you mind adding it to the wiki here?

https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines

So we don't lose track of it - we don't have the git/gerrit repository up
quite yet.

Regards,

Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] Request Validation - Stoplight

[Christopher Yeoh]

On Mon, 20 Oct 2014 10:38:58 -0400
Jay Pipes  wrote:
> 
> > For stackers who are interested in different validation frameworks
> > to implement validation, I recommend checking out Stoplight.
> 
> Just my two cents on this particular topic, I think it's more
> important to standardize ways in which our public REST APIs expose
> the payload expectations and response schemas to clients. In other
> words... we need to focus on methods for API discovery. Once you have
> standardized resource URI, request payload, and response schema
> discovery, then any number of validation libraries may be used.
> 

I agree standardising our APIs is more important. However there is an
advantage to projects using the same libraries where practical. Its
easier for developers to move from project to project, fewer
dependencies for the project overall and when we do run into problems
with libraries there are more people familiar with library. 

However I don't think we should be mandating specific libraries, but we
can make recommendations (good or bad) based on actual experience. This
will be especially useful to new projects starting up to benefit from
the pain other projects have experienced.

Regards,

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] APIImpact flag for nova specs

[Christopher Yeoh]

On Mon, 20 Oct 2014 14:44:15 -0500
Anne Gentle  wrote:
> >
> >
> I think adding APIImpact will be useful.
> 
> I also want to point to the addition of Compute v2 (haven't yet
> proposed a spec for v2.1) to the nova-specs repo here:
> 
> https://review.openstack.org/#/c/129329/
> 
> The goal is to move information from the compute-api repo into the
> -specs repo. I sent an email to the PTLs in August and then added it
> in the What's Up Doc Oct 7th so hopefully this doesn't take anyone by
> surprise. You'll notice if you review that I don't have the template
> test in place that exist for the other blueprint templates, rather
> the ideal is that a new file would be added into api/v2.1 if a
> blueprint affects the API design that describes the correct
> response/request, error codes, and other relevant info. Also if the
> feature affects faults, limits, links, pagination, and so on, the
> spec review would address that in the api spec.

I think this would be a good thing to do. I've added some comments on
the review.

Also here is the review for adding a requirement for an APIImpact
flag in the commit message:

https://review.openstack.org/#/c/129757/

Regards,

Chris

> 
> 
> > Regards,
> >
> > Chris
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] API recommendation

[Christopher Yeoh]

On Thu, Oct 16, 2014 at 7:19 AM, Kevin L. Mitchell <
kevin.mitch...@rackspace.com> wrote:

> On Wed, 2014-10-15 at 12:39 -0400, Andrew Laski wrote:
> > On 10/15/2014 11:49 AM, Kevin L. Mitchell wrote:
> > > Now that we have an API working group forming, I'd like to kick off
> some
> > > discussion over one point I'd really like to see our APIs using (and
> > > I'll probably drop it in to the repo once that gets fully set up): the
> > > difference between synchronous and asynchronous operations.  Using nova
> > > as an example—right now, if you kick off a long-running operation, such
> > > as a server create or a reboot, you watch the resource itself to
> > > determine the status of the operation.  What I'd like to propose is
> that
> > > future APIs use a separate "operation" resource to track status
> > > information on the particular operation.  For instance, if we were to
> > > rebuild the nova API with this idea in mind, booting a new server would
> > > give you a server handle and an operation handle; querying the server
> > > resource would give you summary information about the state of the
> > > server (running, not running) and pending operations, while querying
> the
> > > operation would give you detailed information about the status of the
> > > operation.  As another example, issuing a reboot would give you the
> > > operation handle; you'd see the operation in a queue on the server
> > > resource, but the actual state of the operation itself would be listed
> > > on that operation.  As a side effect, this would allow us (not require,
> > > though) to queue up operations on a resource, and allow us to cancel an
> > > operation that has not yet been started.
> > >
> > > Thoughts?
> >
> > Something like https://review.openstack.org/#/c/86938/ ?
> >
> > I know that Jay has proposed a similar thing before as well.  I would
> > love to get some feedback from others on this as it's something I'm
> > going to propose for Nova in Kilo.
>
> Yep, something very much like that :)  But the idea behind my proposal
> is to make that a codified API guideline, rather than just an addition
> to Nova.
>

Perhaps the best way to make this move faster is for developers not from
Nova
who are interested to help develop the tasks api spec Andrew pointed to.
Its been
 on the Nova to-do list for a few cycles now and had quite a bit of
discussion both
at mid cycles and summit meetings.

Once we have a nova spec approved we can extract the project common parts
out into the API guidelines.

I think we really want microversions up so we can make backwards
incompatible API
changes when we implement the API side of tasks, but that is something
members
of the API WG are hopefully interested in too.

https://review.openstack.org/127127

Regards,

Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] APIImpact flag for nova specs

[Christopher Yeoh]

On Wed, Oct 15, 2014 at 8:58 PM, Sylvain Bauza  wrote:

>
> Le 15/10/2014 11:56, Christopher Yeoh a écrit :
>
>
> On Wed, Oct 15, 2014 at 7:31 PM, Alex Xu  wrote:
>
>>  On 2014年10月15日 14:20, Christopher Yeoh wrote:
>>
>>> Hi,
>>>
>>> I was wondering what people thought of having a convention of adding
>>> an APIImpact flag to proposed nova specs commit messages where the
>>> Nova API will change? It would make it much easier to find proposed
>>> specs which affect the API as its not always clear from the gerrit
>>> summary listing.
>>>
>>  +1, and is there any tool can be used by search flag?
>>
>>>
>>>
>  Can use the message: filter in the gerrit web search interface to search
> in commit messages, or
> alternatively use gerritlib to write something custom.
>
>
> IMHO, asking people to put a tag on a commit msg is good but errorprone
> because there could be some misses.
> Considering that API changes require new templates, why not asking for
> people to provide on a separate tpl file the changes they want to provide,
> and make use of the Gerrit file pattern search like
> specs/kilo/approved/*.tpl ?
>
>
>
We don't require new templates as part of nova-specs and api changes don't
necessarily change the api sample tpl files. We do ask for some jsonschema
descriptions of the new APIs input but they work pretty well in the spec
document itself. I agree it could be prone to spelling mistakes etc, though
just being able to search for 'api' would be sufficient and people who
review specs could pick up missing or mispelled flags in the commit message
(and it wouldn't necessarily need to be restricted to just APIImpact as
possible flags).

Regards,

Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] APIImpact flag for nova specs

[Christopher Yeoh]

On Wed, Oct 15, 2014 at 7:31 PM, Alex Xu  wrote:

> On 2014年10月15日 14:20, Christopher Yeoh wrote:
>
>> Hi,
>>
>> I was wondering what people thought of having a convention of adding
>> an APIImpact flag to proposed nova specs commit messages where the
>> Nova API will change? It would make it much easier to find proposed
>> specs which affect the API as its not always clear from the gerrit
>> summary listing.
>>
> +1, and is there any tool can be used by search flag?
>
>>
>>
Can use the message: filter in the gerrit web search interface to search in
commit messages, or
alternatively use gerritlib to write something custom.

Regards,

Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] APIImpact flag for nova specs

[Christopher Yeoh]

Hi,

I was wondering what people thought of having a convention of adding
an APIImpact flag to proposed nova specs commit messages where the
Nova API will change? It would make it much easier to find proposed
specs which affect the API as its not always clear from the gerrit
summary listing.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] Forming the API Working Group

[Christopher Yeoh]

On Tue, 14 Oct 2014 09:45:44 -0400
Jay Pipes  wrote:

> On 10/14/2014 12:52 AM, Christopher Yeoh wrote:
> > On Mon, 13 Oct 2014 22:20:32 -0400
> > Jay Pipes  wrote:
> >
> >> On 10/13/2014 07:11 PM, Christopher Yeoh wrote:
> >>> On Mon, 13 Oct 2014 10:52:26 -0400
> > And whilst I don't have a problem with having some guidelines which
> > suggest a future standard for APIs, I don't think we should be
> > requiring any type of feature which has not yet been implemented in
> > at least one, preferably two openstack projects and released and
> > tested for a cycle. Eg standards should be lagging rather than
> > leading.
> 
> What about "features" in some of our APIs that are *not* preferable?
> For instance: API extensions.
> 
> I think we've seen where API extensions leads us. And it isn't
> pretty. Would you suggest we document what a Nova API extension or a
> Neutron API extension looks like and then propose, for instance, not
> to ever do it again in future APIs and instead use schema
> discoverability?

So if we had standards leading development rather than lagging in the
past then API extensions would have ended up in the standard because we
once thought they were a good idea.

Perhaps we should distinguish in the documentation between
recommendations (future looking) and standards (proven it works well
for us). The latter would be potentially enforced a lot more strictly
than the former.

In the case of extensions I think we should have a section documenting
why we think they're a bad idea and new projects certainly shouldn't
use them. But also give some advice around if they are used what
features they should have (eg version numbers!). Given the time that it
takes to make major API infrastructure changes it is inevitable that
there will be api extensions added in the short to medium term. Because
API development will not just stop while API infrastructure is improved.

> > I think it will be better in git (but we also need it in gerrit)
> > when it comes to resolving conflicts and after we've established a
> > decent document (eg when we have more content). I'm just looking to
> > make it as easy as possible for anyone to add any guidelines now.
> > Once we've actually got something to discuss then we use git/gerrit
> > with patches proposed to resolve conflicts within the document.
> 
> Of course it would be in Gerrit. I just put it up on GitHub first 
> because I can't just add a repo into the openstack/ code
> namespace... :)

I've submitted a patch to add an api-wg project using your repository
as the initial content for the git repository.

https://review.openstack.org/#/c/128466/

Regards,

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] Forming the API Working Group

[Christopher Yeoh]

On Tue, 14 Oct 2014 10:29:34 -0500
Lance Bragstad  wrote:

> I found a couple of free times available for a weekly meeting if
> people are interested:
> 
> https://review.openstack.org/#/c/128332/2
> 
> Not sure if a meeting time has been hashed out already or not, and if
> it has I'll change the patch accordingly. If not, we can iterate on
> possible meeting times in the review if needed. This was to just get
> the ball rolling if we want a weekly meeting. I proposed one review
> for Thursdays at 2100 and a second patch set for 2000, UTC. That can
> easily change, but those were two times that didn't conflict with the
> existing meeting schedules in #openstack-meeting.

So UTC 2000 is 7am Sydney, 5am Tokyo and 4am Beijing time which is
pretty early in the day. I'd suggest UTC  if that's not too late for
others who'd like to participate.

Chris


> 
> 
> 
> On Tue, Oct 14, 2014 at 3:55 AM, Thierry Carrez
>  wrote:
> 
> > Jay Pipes wrote:
> > > On 10/13/2014 07:11 PM, Christopher Yeoh wrote:
> > >> I guess we could also start fleshing out in the repo how we'll
> > >> work in practice too (eg once the document is stable what
> > >> process do we have for making changes - two +2's is probably not
> > >> adequate for something like this).
> > >
> > > We can make it work exactly like the openstack/governance repo,
> > > where ttx has the only ability to +2/+W approve a patch for
> > > merging, and he tallies a majority vote from the TC members, who
> > > vote -1 or +1 on a proposed patch.
> > >
> > > Instead of ttx, though, we can have an API working group lead
> > > selected from the set of folks currently listed as committed to
> > > the effort?
> >
> > Yes, the working group should select a "chair" who would fill the
> > same role I do for the TC (organize meetings, push agenda, tally
> > votes on reviews, etc.)
> >
> > That would be very helpful in keeping that diverse group on track.
> > Now you just need some volunteer :)
> >
> > --
> > Thierry Carrez (ttx)
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] Forming the API Working Group

[Christopher Yeoh]

On Tue, 14 Oct 2014 09:57:01 -0400
Jay Pipes  wrote:
> On 10/14/2014 05:04 AM, Alex Xu wrote:
> > There is one reason to think about what projects *currently* do.
> > When we choice which convention we want.
> > For example, the CamelCase and snake_case, if the most project use
> > snake_case, then choice snake_case style
> > will be the right.
> 
> I would posit that the reason we have such inconsistencies in our 
> project's APIs is that we haven't taken a stand and said "this is the 
> way it must be".

Not only have we not taken a stand, but we haven't actually told anyone
what they should do in the first place. I don't think its defiance on
the part of the developers of the API its just that they don't know
and without any guidance just do what they think is reasonable. And a
lot of the decisions (eg project vs tenant or instance vs server) are
fairly arbitrary - we just need to choose one.

> 
> There's lots of examples of inconsistencies out in the OpenStack
> APIs. We can certainly use a wiki or etherpad page to document those 
> inconsistencies. But, eventually, this working group should produce 
> solid decisions that should be enforced across *future* OpenStack
> APIs. And that guidance should be forthcoming in the next month or
> so, not in one or two release cycles.

I agree.

> 
> I personally think proposing patches to an openstack-api repository
> is the most effective way to make those proposals. Etherpads and wiki
> pages are fine for dumping content, but IMO, we don't need to dump
> content -- we already have plenty of it. We need to propose
> guidelines for *new* APIs to follow.

ok. I'm happy to go that way - I guess I disagree about us already
having a lot of content. I haven't yet seen (maybe I've missed them)
API guidelines suggestions from most of the openstack projects. But
lets just get started :-)

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] Forming the API Working Group

[Christopher Yeoh]

On Tue, 14 Oct 2014 05:27:21 +0200
"Ken'ichi Ohmichi"  wrote:

> 2014-10-13 16:52 GMT+02:00 Jay Pipes :
> > On 10/10/2014 02:05 AM, Christopher Yeoh wrote:
> >>
> >> I agree with what you've written on the wiki page. I think our
> >> priority needs to be to flesh out
> >> https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines
> >> so we have something to reference when reviewing specs. At the
> >> moment I see that document as something anyone should be able to
> >> document a project's API convention even if they conflict with
> >> another project for the moment. Once we've got a fair amount of
> >> content we can start as a group resolving
> >> any conflicts.
> >
> >
> > Agreed that we should be fleshing out the above wiki page. How
> > would you like us to do that? Should we have an etherpad to discuss
> > individual topics? Having multiple people editing the wiki page
> > offering commentary seems a bit chaotic, and I think we would do
> > well to have the Gerrit review process in place to handle proposed
> > guidelines and rules for APIs. See below for specifics on this...
> >
> >> Speaking of the wiki page, I wrote it very matter-of-factly.
> >> As if this is the way things are. They’re not. The wiki page is
> >> just a starting point. If something was missed, add it. If
> >> something can be improved, improve it. Let’s try to keep it simple
> >> though.
> >>
> >> One problem with API WG members reviewing spec proposals that
> >> affect the API is finding the specs in the first place across many
> >> different projects repositories.
> >
> >
> > I've said for a while now that I would love to have separate
> > repositories -- ala the Keystone API in the openstack/identity-api
> > repository -- that contains specifications for APIs in a single
> > format (APIBlueprint was suggested at one point, but Swagger 2.0
> > seems to me to have more upside).
> >
> > I also think it would be ideal to have an openstack/openstack-api
> > repo that would house guidelines and rules that this working group
> > came up with, along with examples of appropriate usage. This repo
> > would function very similar to the openstack/governance [1] repo
> > that the TC uses to flesh out proposals on community, release
> > management, and governance changes.
> >
> > If people are OK with this idea, I will go ahead and create the
> > repo and add the wiki page content as the initial commit, then
> > everyone can simply submit patches to the document(s) using the
> > normal Gerrit process, and we can iterate on these things using the
> > same tools as other repositories.
> 
> Thanks Jay,
> 
> I much prefer this idea.
> I concerned how to handle API rule conflicts if using a wiki page.
> eg: Someone prefer CamelCase names as attributes but the other does
> snake_case. If using gerrit, we can propose favorite rules as each
> commit and we can discuss them on it. That would be nice to build a
> consensus for the rules.

So for now I'd suggest just adding both - CamelCase and snake_case as
possible guidelines. Once we have all these (possibly conflicting) ideas
in the document then we move it to git/gerrit and propose patches to
resolve any conflicts or remove bits that people don't like.

Eg. I'm suggesting we work from a large document where everyone gets to
throw their opinions in and cut it down rather than start from nothing
and try to build it up patch by patch. I think its a better way for
ensuring people see other people's ideas and where they conflict with
other people's.

Regards,

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] Forming the API Working Group

[Christopher Yeoh]

On Mon, 13 Oct 2014 22:20:32 -0400
Jay Pipes  wrote:

> On 10/13/2014 07:11 PM, Christopher Yeoh wrote:
> > On Mon, 13 Oct 2014 10:52:26 -0400
> > Jay Pipes  wrote:
> >
> >> On 10/10/2014 02:05 AM, Christopher Yeoh wrote:
> >>> I agree with what you've written on the wiki page. I think our
> >>> priority needs to be to flesh out
> >>> https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines
> >>> so we have something to reference when reviewing specs. At the
> >>> moment I see that document as something anyone should be able to
> >>> document a project's API convention even if they conflict with
> >>> another project for the moment. Once we've got a fair amount of
> >>> content we can start as a group resolving
> >>> any conflicts.
> >>
> >> Agreed that we should be fleshing out the above wiki page. How
> >> would you like us to do that? Should we have an etherpad to discuss
> >> individual topics? Having multiple people editing the wiki page
> >> offering commentary seems a bit chaotic, and I think we would do
> >> well to have the Gerrit review process in place to handle proposed
> >> guidelines and rules for APIs. See below for specifics on this...
> >
> > Honestly I don't think we have enough content yet to have much of a
> > discussion. I started the wiki page
> >
> > https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines
> >
> > in the hope that people from other projects would start adding
> > conventions that they use in their projects. I think its fine for
> > the moment if its contradictory, we just need to gather what
> > projects currently do (or want to do) in one place so we can start
> > discussing any contradictions.
> 
> Actually, I don't care all that much about what projects *currently*
> do. I want this API working group to come up with concrete guidelines
> and rules/examples of what APIs *should* look like.

What projects currently do gives us a baseline to work from. It also
should expose where we have currently have inconsistencies between
projects.

And whilst I don't have a problem with having some guidelines which
suggest a future standard for APIs, I don't think we should be
requiring any type of feature which has not yet been implemented in
at least one, preferably two openstack projects and released and tested
for a cycle. Eg standards should be lagging rather than leading.

> > So I'd again encourage anyone interested in APIs from the various
> > projects to just start dumping their project viewpoint in there.
> 
> I went ahead and just created a repository that contained all the
> stuff that should be pretty much agreed-to, and a bunch of stub topic 
> documents that can be used to propose specific ideas (and get
> feedback on) here:
> 
> http://github.com/jaypipes/openstack-api
> 
> Hopefully, you can give it a look and get a feel for why I think the 
> code review process will be better than the wiki for controlling the 
> deliverables produced by this team...

I think it will be better in git (but we also need it in gerrit) when
it comes to resolving conflicts and after we've established a decent
document (eg when we have more content). I'm just looking to make it
as easy as possible for anyone to add any guidelines now. Once we've
actually got something to discuss then we use git/gerrit with patches
proposed to resolve conflicts within the document.

> 
> > I like the idea of a repo and using Gerrit for discussions to
> > resolve issues. I don't think it works so well when people are
> > wanting to dump lots of information in initially.  Unless we agree
> > to just merge anything vaguely reasonable and then resolve the
> > conflicts later when we have a reasonable amount of content.
> > Otherwise stuff will get lost in gerrit history comments and
> > people's updates to the document will overwrite each other.
> >
> > I guess we could also start fleshing out in the repo how we'll work
> > in practice too (eg once the document is stable what process do we
> > have for making changes - two +2's is probably not adequate for
> > something like this).
> 
> We can make it work exactly like the openstack/governance repo, where 
> ttx has the only ability to +2/+W approve a patch for merging, and he 
> tallies a majority vote from the TC members, who vote -1 or +1 on a 
> proposed patch.
> 
> Instead of ttx, though, we can have an API working group lead
> selected from the set of folks currently listed as committed to the
> effort?

Yep, that sounds fine, though I don't think a simple majority is
sufficient for something like api standards. We either get consensus
or we don't include it in the final document.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] Forming the API Working Group

[Christopher Yeoh]

On Mon, 13 Oct 2014 10:52:26 -0400
Jay Pipes  wrote:

> On 10/10/2014 02:05 AM, Christopher Yeoh wrote:
> > I agree with what you've written on the wiki page. I think our
> > priority needs to be to flesh out
> > https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines
> > so we have something to reference when reviewing specs. At the
> > moment I see that document as something anyone should be able to
> > document a project's API convention even if they conflict with
> > another project for the moment. Once we've got a fair amount of
> > content we can start as a group resolving
> > any conflicts.
> 
> Agreed that we should be fleshing out the above wiki page. How would
> you like us to do that? Should we have an etherpad to discuss
> individual topics? Having multiple people editing the wiki page
> offering commentary seems a bit chaotic, and I think we would do well
> to have the Gerrit review process in place to handle proposed
> guidelines and rules for APIs. See below for specifics on this...

Honestly I don't think we have enough content yet to have much of a
discussion. I started the wiki page

https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines

in the hope that people from other projects would start adding
conventions that they use in their projects. I think its fine for the
moment if its contradictory, we just need to gather what projects
currently do (or want to do) in one place so we can start discussing any
contradictions.

So I'd again encourage anyone interested in APIs from the various
projects to just start dumping their project viewpoint in there.

> > Speaking of the wiki page, I wrote it very matter-of-factly. As
> > if this is the way things are. They’re not. The wiki page is just a
> > starting point. If something was missed, add it. If something
> > can be improved, improve it. Let’s try to keep it simple though.
> >
> > One problem with API WG members reviewing spec proposals that
> > affect the API is finding the specs in the first place across many
> > different projects repositories.
> 
> I've said for a while now that I would love to have separate 
> repositories -- ala the Keystone API in the openstack/identity-api 
> repository -- that contains specifications for APIs in a single
> format (APIBlueprint was suggested at one point, but Swagger 2.0
> seems to me to have more upside).
> 
> I also think it would be ideal to have an openstack/openstack-api
> repo that would house guidelines and rules that this working group
> came up with, along with examples of appropriate usage. This repo
> would function very similar to the openstack/governance [1] repo that
> the TC uses to flesh out proposals on community, release management,
> and governance changes.
> 
> If people are OK with this idea, I will go ahead and create the repo
> and add the wiki page content as the initial commit, then everyone
> can simply submit patches to the document(s) using the normal Gerrit 
> process, and we can iterate on these things using the same tools as 
> other repositories.

I like the idea of a repo and using Gerrit for discussions to resolve
issues. I don't think it works so well when people are wanting to dump
lots of information in initially.  Unless we agree to just merge
anything vaguely reasonable and then resolve the conflicts later when
we have a reasonable amount of content. Otherwise stuff will get lost in
gerrit history comments and people's updates to the document will
overwrite each other.

I guess we could also start fleshing out in the repo how we'll work in
practice too (eg once the document is stable what process do we have
for making changes - two +2's is probably not adequate for something like this).

Regards,

Chris

> 
> Best,
> -jay
> 
> [1] 
> https://review.openstack.org/#/q/status:open+project:openstack/governance,n,z
> 
> > I invite everyone who chimed in on the original thread [1] that
> > kicked this off to add themselves as a member committed to
> > making the OpenStack APIs better. I’ve Cc’d everyone who asked to
> > be kept in the loop.
> >
> > I already see some cross project summit topics [2] on APIs. But
> > frankly, with the number of people committed to this topic, I’d
> > expect there to be more. I encourage everyone to submit more API
> > related sessions with better descriptions and goals about what
> > you want to achieve in those sessions.
> >
> > Yea if there is enough content in the API guidelines then perhaps
> > some time can be spent on working on resolving any conflicts in the
> > document so projects know what direction to head in.
> >
> > Re

Re: [openstack-dev] [api] Forming the API Working Group

[Christopher Yeoh]

Hi Everett,

Great to see things moving with the API Working Group!

On Thu, Oct 9, 2014 at 9:35 AM, Everett Toews 
wrote:

> https://wiki.openstack.org/wiki/API_Working_Group
>
> This is the start of the API Working Group (API WG).
>
> To avoid bike shedding over the name of the working group, I decided to
> title the wiki page API Working Group. Simple, to the point, and avoids
> loaded terms like standards, best practices, guidelines, conventions, etc.
>
> The point isn’t what we name it. The point is what action we take about
> it. I propose the deliverables in the API WG wiki page.
>
>
I agree with what you've written on the wiki page. I think our priority
needs to be to flesh out
https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines
so we have something to reference when reviewing specs. At the moment I see
that document as something anyone should be able to document a project's
API convention even if they conflict with another project for the moment.
Once we've got a fair amount of content we can start as a group resolving
any conflicts.



> Speaking of the wiki page, I wrote it very matter-of-factly. As if this is
> the way things are. They’re not. The wiki page is just a starting point. If
> something was missed, add it. If something can be improved, improve it.
> Let’s try to keep it simple though.
>
>
One problem with API WG members reviewing spec proposals that affect the
API is finding the specs in the first place across many different projects
repositories.


> I invite everyone who chimed in on the original thread [1] that kicked
> this off to add themselves as a member committed to making the OpenStack
> APIs better. I’ve Cc’d everyone who asked to be kept in the loop.
>
> I already see some cross project summit topics [2] on APIs. But frankly,
> with the number of people committed to this topic, I’d expect there to be
> more. I encourage everyone to submit more API related sessions with better
> descriptions and goals about what you want to achieve in those sessions.
>
>
Yea if there is enough content in the API guidelines then perhaps some time
can be spent on working on resolving any conflicts in the document so
projects know what direction to head in.

Regards,

 Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Nova API meeting

[Christopher Yeoh]

Hi,

Just a reminder that the weekly Nova API meeting is being held tomorrow
Friday UTC . 

We encourage cloud operators and those who use the REST API such as
SDK developers and others who and are interested in the future of the
API to participate.

In other timezones the meeting is at:

EST 20:00 (Thu)
Japan 09:00 (Fri)
China 08:00 (Fri)
ACDT 9:30 (Fri)

The proposed agenda and meeting details are here: 

https://wiki.openstack.org/wiki/Meetings/NovaAPI

Please feel free to add items to the agenda. 

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][docs][tc] How to scale Documentation

[Christopher Yeoh]

On Mon, 06 Oct 2014 11:24:30 +0200
Julien Danjou  wrote:

> On Sun, Oct 05 2014, Joshua Harlow wrote:
> 
> I agree. I think we should have documentation be part of our policy to
> accept patches, just as we do with e.g. unit testing.
> 
> Forcing people to write documentation along the patch will help
> writing better code, having a better understanding of what the patch
> does for reviewers, etc…
> 

Agreed. And a lot of the confusion we get between what is documented to
happen and what is implemented in the code is due to different people
writing the documentation versus writing the code. And there has been
insufficient information (other than the code) left for people to write
accurate documentation.

For API docs I wonder if we should bringing more of that into the
project tree so we can enforce updating of documentation when the code
gets updated.

Chris


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] why do we have os-attach-interfaces in the v3 API?

[Christopher Yeoh]

On Thu, 02 Oct 2014 15:57:55 -0500
Matt Riedemann  wrote:

> The os-interface (v2) and os-attach-interfaces (v3) APIs are only
> used for the neutron network API, you'll get a NotImplemented if
> trying to call the related methods with nova-network [1].
> 
> Since we aren't proxying to neutron in the v3 API (v2.1), why does 
> os-attach-interfaces [2] exist?  Was this just an oversight?  If so, 
> please allow me to delete it. :)

The proxying work was not done in Juno due to time constraints, but I
think we should be able to cover it early in Kilo (most of the patches
are pretty much ready). To have a V2.1 which is functionality
equivalent to V2 (allowing us to remove the V2 code) we have to
implement proxying.

Chris

> 
> [1] 
> http://git.openstack.org/cgit/openstack/nova/tree/nova/network/api.py?id=2014.2.rc1#n310
> [2] 
> http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/plugins/v3/attach_interfaces.py?id=2014.2.rc1
> 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] No Nova API meeting this week

[Christopher Yeoh]

Hi,

About half the regulars at the Nova API meeting are on vacation
this week so I'm cancelling this weeks meeting as I don't
think there is anything urgent to discuss. We'll meet as usual
next week.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] Fate of xmlutils

[Christopher Yeoh]

On Mon, 29 Sep 2014 18:03:20 +0200
Julien Danjou  wrote:
> 
> It seems that Python fixed that issue with 2 modules released on PyPI:
> 
>   https://pypi.python.org/pypi/defusedxml
>   https://pypi.python.org/pypi/defusedexpat
> 
> I'm no XML expert, and I've only a shallow understanding of the issue,
> but I wonder if we should put some efforts to drop xmlutils and our
> custom XML fixes to used instead these 2 modules.

Nova XML API support is marked as deprecated in Juno. So hopefully
we'll be able to just drop XML and associated helper modules within a
couple of cycles.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Kilo Blueprints and Specs

[Christopher Yeoh]

On Mon, 29 Sep 2014 13:32:57 -0700
Joe Gordon  wrote:

> On Mon, Sep 29, 2014 at 5:23 AM, Gary Kotton 
> wrote:
> 
> > Hi,
> > Is the process documented anywhere? That is, if say for example I
> > had a spec approved in J and its code did not land, how do we go
> > about kicking the tires for K on that spec.
> >
> 
> Specs will need be re-submitted once we open up the specs repo for
> Kilo. The Kilo template will be changing a little bit, so specs will
> need a little bit of reworking. But I expect the process to approve
> previously approved specs to be quicker

Am biased given I have a spec approved for Juno which we didn't quite
fully merge which we want to finish off early in Kilo (most of the
patches are very close already to being ready to merge), but I think we
should give priority to reviewing specs already approved in Juno and
perhaps only require one +2 for re-approval. 

Otherwise we'll end up wasting weeks of development time just when
there is lots of review bandwidth available and the CI system is
lightly loaded. Honestly, ideally I'd like to just start merging as
soon as Kilo opens. Nothing has changed between Juno FF and Kilo opening
so there's really no reason that an approved Juno spec should not be
reapproved.

Chris

> 
> 
> > Thanks
> > Gary
> >
> > On 9/29/14, 1:07 PM, "John Garbutt"  wrote:
> >
> > >On 27 September 2014 00:31, Joe Gordon 
> > >wrote:
> > >> On Thu, Sep 25, 2014 at 9:21 AM, John Garbutt
> > >> 
> > >>wrote:
> > >>> On 25 September 2014 14:10, Daniel P. Berrange
> > >>>  wrote:
> > >>> >> The proposal is to keep kilo-1, kilo-2 much the same as juno.
> > >>>Except,
> > >>> >> we work harder on getting people to buy into the priorities
> > >>> >> that are set, and actively provoke more debate on their
> > >>> >> "correctness", and we reduce the bar for what needs a
> > >>> >> blueprint.
> > >>> >>
> > >>> >> We can't have 50 high priority blueprints, it doesn't mean
> > >>> >> anything, right? We need to trim the list down to a
> > >>> >> manageable number, based
> > >>>on
> > >>> >> the agreed project priorities. Thats all I mean by slots /
> > >>> >> runway at this point.
> > >>> >
> > >>> > I would suggest we don't try to rank high/medium/low as that
> > >>> > is too coarse, but rather just an ordered priority list. Then
> > >>> > you would not be in the situation of having 50 high
> > >>> > blueprints. We would instead naturally just start at the
> > >>> > highest priority and work downwards.
> > >>>
> > >>> OK. I guess I was fixating about fitting things into launchpad.
> > >>>
> > >>> I guess having both might be what happens.
> > >>>
> > >>> >> > The runways
> > >>> >> > idea is just going to make me less efficient at reviewing.
> > >>> >> > So I'm very much against it as an idea.
> > >>> >>
> > >>> >> This proposal is different to the runways idea, although it
> > >>>certainly
> > >>> >> borrows aspects of it. I just don't understand how this
> > >>> >> proposal has all the same issues?
> > >>> >>
> > >>> >>
> > >>> >> The key to the kilo-3 proposal, is about getting better at
> > >>> >> saying
> > >>>no,
> > >>> >> this blueprint isn't very likely to make kilo.
> > >>> >>
> > >>> >> If we focus on a smaller number of blueprints to review, we
> > >>> >> should
> > >>>be
> > >>> >> able to get a greater percentage of those fully completed.
> > >>> >>
> > >>> >> I am just using slots/runway-like ideas to help pick the high
> > >>>priority
> > >>> >> blueprints we should concentrate on, during that final
> > >>> >> milestone. Rather than keeping the distraction of 15 or so
> > >>> >> low priority blueprints, with those poor submitters jamming
> > >>> >> up the check queue,
> > >>>and
> > >>> >> constantly rebasing, and having to deal with the odd stray
> > >>> >> review comment they might get lucky enough to get.
> > >>> >>
> > >>> >> Maybe you think this bit is overkill, and thats fine. But I
> > >>> >> still think we need a way to stop wasting so much of peoples
> > >>> >> time on
> > >>>things
> > >>> >> that will not make it.
> > >>> >
> > >>> > The high priority blueprints are going to end up being mostly
> > >>> > the big scope changes which take alot of time to review &
> > >>> > probably go through many iterations. The low priority
> > >>> > blueprints are going to end up
> > >>>being
> > >>> > the small things that don't consume significant resource to
> > >>> > review
> > >>>and
> > >>> > are easy to deal with in the time we're waiting for the big
> > >>> > items to go through rebases or whatever. So what I don't like
> > >>> > about the
> > >>>runways
> > >>> > slots idea is that removes the ability to be agile and take
> > >>> > the initiative
> > >>> > to review & approve the low priority stuff that would
> > >>> > otherwise never make it through.
> > >>>
> > >>> The idea is more around concentrating on the *same* list of
> > >>> things.
> > >>>
> > >>> Certainly we need to avoid the priority inversion of
> > >>> concentrating only on the big things.
> 

Re: [openstack-dev] [NOVA] security group fails to attach to an instance if port-id is specified during boot.

[Christopher Yeoh]

On Fri, 26 Sep 2014 11:25:49 +0400
Oleg Bondarev  wrote:

> On Fri, Sep 26, 2014 at 3:30 AM, Day, Phil  wrote:
> 
> >  I think the expectation is that if a user is already interaction
> > with Neutron to create ports then they should do the security group
> > assignment in Neutron as well.
> >
> 
> Agree. However what do you think a user expects when he/she boots a
> vm (no matter providing port_id or just net_id)
> and specifies security_groups? I think the expectation should be that
> instance will become a member of the specified groups.
> Ignoring security_groups parameter in case port is provided (as it is
> now) seems completely unfair to me.

One option would be to return a 400 if both port id and security_groups
is supplied.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] logging around olso lockutils

[Christopher Yeoh]

On Thu, 25 Sep 2014 08:49:12 -0400
Sean Dague  wrote:

> 
> #1 - tried to get a lock, but someone else has it. Then we know we've
> got lock contention. .
> #2 - something is still holding a lock after some "long" amount of
> time.

+1 to both.

> #2 turned out to be a critical bit in understanding one of the worst
> recent gate impacting issues.
> 
> You can write a tool today that analyzes the logs and shows you these
> things. However, I wonder if we could actually do something creative
> in the code itself to do this already. I'm curious if the creative
> use of Timers might let us emit log messages under the conditions
> above (someone with better understanding of python internals needs to
> speak up here). Maybe it's too much overhead, but I think it's worth
> at least asking the question.

Even a simple log at the end when its finally released if it has been
held for a long time would be handy. As matching up acquire/release by
eye is not easy.

I don't think we get a log message when an acquire is attempted but
fails. That might help get a measure of lock contention?

> The same issue exists when it comes to processutils I think, warning
> that a command is still running after 10s might be really handy,
> because it turns out that issue #2 was caused by this, and it took
> quite a bit of decoding to figure that out.

Also I think that perhaps a log message when a period task takes longer
than the interval that the task is meant to be run would be a useful
warning sign that something odd is going on.

Chris


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Nova API meeting

[Christopher Yeoh]

Hi,

Just a reminder that the weekly Nova API meeting is being held tomorrow
Friday UTC . 

We encourage cloud operators and those who use the REST API such as
SDK developers and others who and are interested in the future of the
API to participate.

In other timezones the meeting is at:

EST 20:00 (Thu)
Japan 09:00 (Fri)
China 08:00 (Fri)
ACDT 9:30 (Fri)

The proposed agenda and meeting details are here: 

https://wiki.openstack.org/wiki/Meetings/NovaAPI

Please feel free to add items to the agenda. 

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] [All] API standards working group

[Christopher Yeoh]

On Wed, 24 Sep 2014 13:05:09 -0400
Jay Pipes  wrote:

> On 09/24/2014 12:48 PM, Everett Toews wrote:
> > On Sep 24, 2014, at 9:42 AM, Dean Troyer  wrote:
> >
> >> I'll bring an API consumer's perspective.
> >
> > +1
> >
> > I’d bring an API consumer’s perspective as well.
> >
> > Looks like there’s lots of support for an API WG. What’s the next
> > step?
> >
> > Form a WG under the User Committee [1] or is there something more
> > appropriate?
> >
> > Thanks,
> > Everett
> >
> > [1]
> > https://wiki.openstack.org/wiki/Governance/Foundation/UserCommittee#Working_Groups
> 
> Whatever it ends up being, it needs to have some teeth to it.
> Otherwise, we're going to end up in the exact same place we're in
> now, where each project does something slightly different.

I think its more an issue around actually having guidelines and
educating people where to find them than needing teeth to enforce them.
Where I've seen variation in the APIs in Nova its been primarily because
people haven't known what the conventions are than anything else. 

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] [All] API standards working group

[Christopher Yeoh]

On Wed, 24 Sep 2014 16:48:43 +
Everett Toews  wrote:

> On Sep 24, 2014, at 9:42 AM, Dean Troyer  wrote:
> 
> > I'll bring an API consumer's perspective.  
> 
> +1
> 
> I’d bring an API consumer’s perspective as well.
> 
> Looks like there’s lots of support for an API WG. What’s the next
> step?

We have the start of some guidelines here:

https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines

I think it would be very useful if those who are interested in helping
out add info to that wiki from the perspective of the project they work
on and comment (inline) where what is already there is inconsistent with
what they do. User input too as to what they want would be useful. 

That would give us something much more concrete to discuss and debate
rather than just a common desire to have consistency across our APIs.

Regards,

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Christopher Yeoh]

On Wed, 24 Sep 2014 09:04:51 -0400
Jay Pipes  wrote:

> On 09/24/2014 05:26 AM, Kenichi Oomichi wrote:
> >> -Original Message-
> >> From: Jay Pipes [mailto:jaypi...@gmail.com]
> >> Sent: Wednesday, September 24, 2014 12:47 AM
> >> To: openstack-dev@lists.openstack.org
> >> Subject: Re: [openstack-dev] [Nova] Some ideas for micro-version
> >> implementation
> >>
> >> On 09/22/2014 04:27 PM, Brant Knudson wrote:
> >>> On Fri, Sep 19, 2014 at 1:39 AM, Alex Xu  >>> > wrote:
> >>>
> >>>  Close to Kilo, it is time to think about what's next for
> >>> nova API. In Kilo, we
> >>>  will continue develop the important feature micro-version.
> >>>
> >>>  In previous v2 on v3 propose, it's include some
> >>> implementations can be used for micro-version.
> >>>  
> >>> (https://review.openstack.org/__#/c/84695/19/specs/juno/v2-on-__v3-api.rst
> >>>  
> >>> )
> >>>  But finally, those implementations was considered too
> >>> complex.
> >>>
> >>>  So I'm try to find out more simple implementation and
> >>> solution for micro-version.
> >>>
> >>>  I wrote down some ideas as blog post at:
> >>>  http://soulxu.github.io/blog/__2014/09/12/one-option-for-__nova-api/
> >>>  
> >>>
> >>>  And for those ideas also already done some POC, you can find
> >>> out in the blog post.
> >>>
> >>>  As discussion in the Nova API meeting, we want to bring it
> >>> up to mail-list to
> >>>  discussion. Hope we can get more idea and option from all
> >>> developers.
> >>>
> >>>  We will appreciate for any comment and suggestion!
> >>>
> >>>  Thanks
> >>>  Alex
> >>>
> >>>
> >>>
> >>> Did you consider JSON Home[1] for this? For Juno we've got JSON
> >>> Home support in Keystone for Identity v3 (Zaqar was using it
> >>> already). We weren't planning to use it for microversioning since
> >>> we weren't planning on doing microversioning, but I think JSON
> >>> Home could be used for this purpose.
> >>>
> >>> Using JSON Home, you'd have relationships that include the
> >>> version, then the client can check the JSON Home document to see
> >>> if the server has support for the relationship the client wants
> >>> to use.
> >>>
> >>> [1] http://tools.ietf.org/html/draft-nottingham-json-home-03
> >>
> >> ++ I used JSON-Home extensively in the Compute API blueprint I put
> >> together a few months ago:
> >>
> >> http://docs.oscomputevnext.apiary.io/
> >
> > vNext seems an interesting idea, I thought the implementation way
> > for Nova a little. "API Route Discoverability" is a nice design,
> > but a root "/" URL will conflict on current "list versions" API.
> > Maybe there would be a workaround.
> 
> Completely agreed, Ken'ichi. The "root" URL that returns the
> JSON-Home doc in the vNext API is actually *after* the version in the
> URI, though...
> 
> So, the JSON-Home doc would be returned from:
> 
>   http://compute.example.com/vNext/
> 
> Of course, replacing "vNext" with "v4" or "v42" or whatever the
> "next" major version of the API would be. The real root would still
> return the versions list as it exists today, with a 302 Multiple
> Choice.

Why have a version in the url at all? With microversions it doesn't
really make any sense and would just cause more churn for app
developers.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] [All] API standards working group

[Christopher Yeoh]

On Tue, 23 Sep 2014 18:18:56 -0400
Jay Pipes  wrote:

> On 09/23/2014 05:03 PM, Rochelle.RochelleGrober wrote:
> > jaypi...@gmail.com  on Tuesday,
> > September 23, 2014 9:09 AM wrote:
> >
> > _Snip
> >
> > I'd like to say finally that I think there should be an OpenStack
> > API
> >
> > working group whose job it is to both pull together a set of
> > OpenStack
> >
> > API practices as well as evaluate new REST APIs proposed in the
> >
> > OpenStack ecosystem to provide guidance to new projects or new
> >
> > subprojects wishing to add resources to an existing REST API.
> >
> > Best,
> >
> > -jay
> >
> > */[Rocky Grober] /*++
> >
> > */Jay, are you volunteering to head up the working group? Or at
> > least be an active member?  I’ll certainly follow with interest,
> > but I think I have my hands full with the log rationalization
> > working group./*
> 
> Yes, I'd be willing to head up the working group... or at least 
> participate in it.


I'd be very interested in participating as well.

At the last Design Summit at the cross project API consistency session
we talked about setting up a wiki page for guidelines. I've made a
start on this though its very much from a Nova perspective and could
do with input from all the other projects.

https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] REST API style guide (Re: [Nova] Some ideas for micro-version implementation)

[Christopher Yeoh]

On Tue, 23 Sep 2014 09:00:26 +0900
"Ken'ichi Ohmichi"  wrote:
\> 
> So how about just using HTTP 200(OK) only for status codes?
> That would give up providing accurate internal status to clients but
> backwards backwards incompatibilities never happen.
>

No I think that we should where possible return the most accurate
status code. A 202 versus 200 is an important distinction for a user of
the API (eg do they need to poll for request completion?). How fast
we can get to accurate status codes through the API is a different
matter though. 

 
> and I have one more idea for making API consistency of whole
> OpenStack projects. That is each rule of the style guide is
> implemented in Tempest. Tempest has its own REST clients for many
> projects and we can customize them for improving qualities. After
> defining the REST API style guide, we can add each
> rule to Tempest's base client class and apply it for all REST APIs
> which are tested
> by Tempest. We can keep consistent API for the existing projects and
> apply the style guide to new projects also by this framework.

That's an interesting idea! However we would have a long exception list
for a quite a while I think as it was made pretty clear to use we can't
make a large number of backwards compatible API changes over a short
period of time. (Like a v2->v3 transition was).

Regards,

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Christopher Yeoh]

On Mon, 22 Sep 2014 15:27:47 -0500
Brant Knudson  wrote:

> >
> >
> Did you consider JSON Home[1] for this? For Juno we've got JSON Home
> support in Keystone for Identity v3 (Zaqar was using it already). We
> weren't planning to use it for microversioning since we weren't
> planning on doing microversioning, but I think JSON Home could be
> used for this purpose.
> 
> Using JSON Home, you'd have relationships that include the version,
> then the client can check the JSON Home document to see if the server
> has support for the relationship the client wants to use.
> 

JSON Home is on our list of things to do for the API (but
the priority is lower than microversions). We still need
microversions anyway because we want to be able to support multiple
versions of API through the same resources (ie no url changes). To
make life as easy as possible for users so they are not forced to
upgrade their app whenever we make backwards incompatible changes. I
don't see us supporting every version forever, but it gives them some
time to upgrade.

Regards,

Chris
 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Christopher Yeoh]

On Mon, 22 Sep 2014 09:47:50 -0500
Anne Gentle  wrote:
> >
> > (1) Input/Output attribute names
> > (1.1) These names should be snake_case.
> >   eg: imageRef -> image_ref, flavorRef -> flavor_ref, hostId ->
> > host_id (1.2) These names should contain extension names if they
> > are provided in case of some extension loading.
> >   eg: security_groups -> os-security-groups:security_groups
> >   config_drive -> os-config-drive:config_drive
> >
> 
> Do you mean that the os- prefix should be dropped? Or that it should
> be maintained and added as needed?


Originally (I think) the os- prefix was added to avoid potential
name clashes between extensions. Presumably out of tree extensions
too as in-tree ones would be picked up pretty quickly. For a while now
I've been thinking that perhaps we should just drop the os- prefix.
We're trying to discourage optional extensions anyway and I suspect that
the pain of maintaining consistency with os- prefixes is not worth the
benefit.

Where if someone wants to maintain an out of tree plugin they
can bear the burden of avoiding name clashes. This would make it much
easier for both us (nearly no code changes required) and users (no app
changes) when we move some functionality into (or out of) core.

> > (1.3) Extension names should consist of hyphens and low chars.
> >   eg: OS-EXT-AZ:availability_zone ->
> > os-extended-availability-zone:availability_zone
> >   OS-EXT-STS:task_state -> os-extended-status:task_state
> >
> 
> Yes, I don't like the shoutyness of the ALL CAPS.

+1! No to ALL CAPS and contractions or disemvowling of words in order
to save a few bytes.

> > (1.4) Extension names should contain the prefix "os-" if the
> > extension is not core.
> >   eg: rxtx_factor -> os-flavor-rxtx:rxtx_factor
> >   os-flavor-access:is_public -> flavor-access:is_public
> > (flavor-access extension became core)
> >
> 
> Do we have a list of "core" yet?

We do sort of have a candidate listed hard coded (its a pretty small
list):

API_V3_CORE_EXTENSIONS = set(['consoles',
  'extensions',
  'flavor-access',
  'flavor-extra-specs',
  'flavor-manage',
  'flavors',
  'ips',
  'os-keypairs',
  'server-metadata',
  'servers',
  'versions'])

> 
> > (3) Status code
> > (3.1) Return 201(Created) if a resource creation/updating finishes
> > before returning a response.
> >   eg: "create a keypair" API: 200 -> 201
> >   "create an agent" API: 200 -> 201
> >   "create an aggregate" API: 200 -> 201
> >
> 
> Do you mean a 200 becomes a 201? That's a huge doc impact and SDK
> impact, is it worthwhile? If we do this change, the sooner the
> better, right?

Ideally I think we'd want to do it when breaking a specific part of the
API anyway (for say some new feature). Otherwise its something I think
we should bring up on a case by case with users and operators. Trade
off between returning misleading status codes versus requiring
application side changes (potentially, some may just look for 2xx
anyway).

> >
> >
> The TC had an action item a while back (a few months) to start an API
> style guide. This seems like a good start. Once the questions are
> discussed I'll get a draft going on the wiki.

So back during the Juno design summit at the cross project API
consistency session we talked about putting together a wiki page with
guidelines for all OpenStack projects. But I think everyone got busy so
not much at all happened. I've had a bit of time recently and tried to
pull together info from the session as well as some other sources and
the start of it is here:

https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines

So perhaps we could merge what is above into this wiki?
It is however still rather Nova specific and we need input from
other projects (I'll send out another email specifically about this). 

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Christopher Yeoh]

On Mon, 22 Sep 2014 09:29:26 +
Kenichi Oomichi  wrote:
> 
> Before discussing how to implement, I'd like to consider what we
> should implement. IIUC, the purpose of v3 API is to make consistent
> API with the backwards incompatible changes. Through huge discussion
> in Juno cycle, we knew that backwards incompatible changes of REST
> API would be huge pain against clients and we should avoid such
> changes as possible. If new APIs which are consistent in Nova API
> only are inconsistent for whole OpenStack projects, maybe we need to
> change them again for whole OpenStack consistency.

So I think there's three different aspects to microversions which we
can consider quit separately:

- The format of the client header and what the version number means.
  Eg is the version number of the format X.Y.Z, what do we increment
  when we make a bug fix, what do we increment when we make a backwards
  compatible change and what do we increment when we make backwards
  incompatible change.

  Also how does a client request experimental APIs (I believe we have
  consensus that we really need this to avoid backwards incompatible
  changes as much as possible as it allows more testing before
  guaranteeing backwards compatibility)

  I believe that we can consider this part separately from the next two
  issues.

- The implementation on the nova api side. Eg how do we cleanly handle
  supporting multiple versions of the api based on the client header
  (or lack of it which will indicate v2 compatibility. I'll respond
  directly on Alex's original post

- What we are going to use the microversions API feature to do. I think
  they fall under a few broad categories: 

  - Backwards compatible changes. We desperately need a mechanism that
allows us to make backwards compatible changes (eg add another
parameter to a response) without having to add another dummy
extension.

  - Significant backwards incompatible changes. The Tasks API and server
diagnostics API are probably the best examples of this. 

  - V3 like backwards incompatible changes (consistency fixes).

I think getting consensus over backwards compatible changes will be
straightforward. However given the previous v2/v3 discussions I don't
think we will be able to get consensus over doing all or most of the
consistency type fixes even using microversions in the short term.
Because with microversions you get all the changes applied before the
version that you choose. So from a client application point of view its
just as much work as V2 to V3 API transition.

I don't think that means we need to put all of these consistency
changes off forever though. We need to make backwards incompatible
changes in order to implement the Tasks API  and new server
diagnostics api the way we want to. The Tasks API will eventually cover
quite a few interfaces and while say breaking backwards compatibility
with the create server api, we can also fix consistency issues in that
api at the same time. Clients will need to make changes to their app
anyway if they want to take advantage of the new features (or they can
just continue to use the old non-tasks enabled API).

So as we slowly make backwards incompatible changes to the API for
other reasons we can also fix up other issues. Other consistency fixes
we can propose on a case by case basis and the user community can have
input as to whether the cost (app rework) is worth it without getting a
new feature at the same time.

But I think its clear that we *need* the microversions mechanism. So we
don't need to decide beforehand exactly what we're going to use it for
first. I think think its more important that we get a nova-spec
approved for the the first two parts - what it looks like from the
client point of view. And how we're going to implement it.

Regards,

Chris

> 
> For avoiding such situation, I think we need to define what is
> consistent REST API across projects. According to Alex's blog, The
> topics might be
> 
>  - Input/Output attribute names
>  - Resource names
>  - Status code
> 
> The following are hints for making consistent APIs from Nova v3 API
> experience, I'd like to know whether they are the best for API
> consistency.
> 
> (1) Input/Output attribute names
> (1.1) These names should be snake_case. 
>   eg: imageRef -> image_ref, flavorRef -> flavor_ref, hostId ->
> host_id (1.2) These names should contain extension names if they are
> provided in case of some extension loading. eg: security_groups ->
> os-security-groups:security_groups config_drive ->
> os-config-drive:config_drive (1.3) Extension names should consist of
> hyphens and low chars. eg: OS-EXT-AZ:availability_zone ->
> os-extended-availability-zone:availability_zone OS-EXT-STS:task_state
> -> os-extended-status:task_state (1.4) Extension names should contain
> the prefix "os-" if the extension is not core. eg: rxtx_factor ->
> os-flavor-rxtx:rxtx_factor os-flavor-access:is_public ->
> flavor-access:is_public (flavor-access exte

Re: [openstack-dev] [nova] Expand resource name allowed characters

[Christopher Yeoh]

On Thu, 18 Sep 2014 12:12:28 -0400
Sean Dague  wrote:
> > When we can return the json-schema to user in the future, can we say
> > that means API accepting utf8 or utf8mb4 is discoverable? If it is
> > discoverable, then we needn't limit anything in our python code.
> 
> Honestly, we should accept utf8 (no weird mysqlism not quite utf8). We
> should make the default scheme for our dbs support that on names (but
> only for the name columns). The failure of a backend to do utf8 for
> real should return an error to the user. Let's not make this more
> complicated than it needs to be.

I agree that discoverability for this is not the way to go - I think its
too complicated for end users. I don't know enough about mysql to know
if utf8mb4 is going to a performance issue but if its not then we
should just support utf-8 properly. 

We can we can catch the db errors. However whilst converting db
errors causing 500s is fairly straightforward when an error occurs that
deep in Nova it also means a lot of potential unwinding work in the db
and compute layers which is complicated and error prone. So i'd prefer
to avoid the situation with input validation in the first place. 

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Expand resource name allowed characters

[Christopher Yeoh]

On Thu, 18 Sep 2014 13:45:42 +0200
Flavio Percoco  wrote:

> On 09/18/2014 01:28 PM, Sean Dague wrote:
> > On 09/18/2014 07:19 AM, Ken'ichi Ohmichi wrote:
> >> 2014-09-18 19:57 GMT+09:00 Sean Dague :
> >>> On 09/18/2014 06:38 AM, Christopher Yeoh wrote:
> >>>> On Sat, 13 Sep 2014 06:48:19 -0400
> >>>> Sean Dague  wrote:
> >>>>>>>
> >>>>>>> We have proposed that the allowed characters for all resource
> >>>>>>> names in Nova (flavors, aggregates, etc.) be expanded to all
> >>>>>>> printable unicode characters and horizontal spaces:
> >>>>>>> https://review.openstack.org/#/c/119741
> >>>>>>>
> >>>>>>> Currently, the only allowed characters in most resource names
> >>>>>>> are alphanumeric, space, and [.-_].
> >>>>>>>
> >>>>>>>
> >>>>>>> We have proposed this change for two principal reasons:
> >>>>>>>
> >>>>>>> 1. We have customers who have migrated data forward since
> >>>>>>> Essex, when no restrictions were in place, and thus have
> >>>>>>> characters in resource names that are disallowed in the
> >>>>>>> current version of OpenStack. This is only likely to be
> >>>>>>> useful to people migrating from Essex or earlier, since the
> >>>>>>> current restrictions were added in Folsom.
> >>>>>>>
> >>>>>>> 2. It's pretty much always a bad idea to add unnecessary
> >>>>>>> restrictions without a good reason. While we don't have an
> >>>>>>> immediate need to use, for example, the ever-useful
> >>>>>>> http://codepoints.net/U+1F4A9 in a flavor name, it's hard to
> >>>>>>> come up with a reason people *shouldn't* be allowed to use it.
> >>>>>>>
> >>>>>>> That said, apparently people have had a need to not be
> >>>>>>> allowed to use some characters, but it's not clear why:
> >>>>>>> https://bugs.launchpad.net/nova/+bug/977187 So I guess if
> >>>>>>> anyone knows any reason why these printable characters should
> >>>>>>> not be joined in holy resource naming, speak now or forever
> >>>>>>> hold your peace.
> >>>>>>
> >>>>>> I also could not find the reason of current restriction on the
> >>>>>> bug report, and I'd like to know it as the history.
> >>>>>> On v2 API(not v2.1), each resource name contains the following
> >>>>>> restriction for its name:
> >>>>>>
> >>>>>>   Resource  | Length  | Pattern
> >>>>>>  ---+-+--
> >>>>>>   aggregate | 1-255   | nothing
> >>>>>>   backup| nothing | nothing
> >>>>>>   flavor| 1-255   | '^[a-zA-Z0-9. _-]*[a-zA-Z0-9_-]+
> >>>>>> | |   [a-zA-Z0-9. _-]*$'
> >>>>>>   keypair   | 1-255   | '^[a-zA-Z0-9 _-]+$'
> >>>>>>   server| 1-255   | nothing
> >>>>>>   cell  | 1-255   | don't contain "." and "!"
> >>>>>>
> >>>>>> On v2.1 API, we have applied the same restriction rule[1] for
> >>>>>> whole resource names for API consistency, so maybe we need to
> >>>>>> consider this topic for whole names.
> >>>>>>
> >>>>>> [1]:
> >>>>>> https://github.com/openstack/nova/blob/master/nova/api/validation/parameter_types.py#L44
> >>>>>
> >>>>> Honestly, I bet this had to do with how the database used to be
> >>>>> set up.
> >>>>>
> >>>>
> >>>> So it turns out that utf8 support in MySQL does not support
> >>>> UTF-8 4 byte multibyte characters (only 1-3 bytes). For example
> >>>> if you do a create image call with an image name to glance with
> >>>> a 4 byte multibyte character in the name it will 500. I'd guess
> >>>> we do something similar in places with the Nova API where we
> >>>> have inadequate input validation. 

Re: [openstack-dev] [nova] Expand resource name allowed characters

[Christopher Yeoh]

On Sat, 13 Sep 2014 06:48:19 -0400
Sean Dague  wrote:

> On 09/13/2014 02:28 AM, Kenichi Oomichi wrote:
> > 
> > Hi Chris,
> > 
> > Thanks for bring it up here,
> > 
> >> -Original Message-
> >> From: Chris St. Pierre [mailto:stpie...@metacloud.com]
> >> Sent: Saturday, September 13, 2014 2:53 AM
> >> To: openstack-dev@lists.openstack.org
> >> Subject: [openstack-dev] [nova] Expand resource name allowed
> >> characters
> >>
> >> We have proposed that the allowed characters for all resource
> >> names in Nova (flavors, aggregates, etc.) be expanded to all
> >> printable unicode characters and horizontal spaces:
> >> https://review.openstack.org/#/c/119741
> >>
> >> Currently, the only allowed characters in most resource names are
> >> alphanumeric, space, and [.-_].
> >>
> >>
> >> We have proposed this change for two principal reasons:
> >>
> >> 1. We have customers who have migrated data forward since Essex,
> >> when no restrictions were in place, and thus have characters in
> >> resource names that are disallowed in the current version of
> >> OpenStack. This is only likely to be useful to people migrating
> >> from Essex or earlier, since the current restrictions were added
> >> in Folsom.
> >>
> >> 2. It's pretty much always a bad idea to add unnecessary
> >> restrictions without a good reason. While we don't have an
> >> immediate need to use, for example, the ever-useful
> >> http://codepoints.net/U+1F4A9 in a flavor name, it's hard to come
> >> up with a reason people *shouldn't* be allowed to use it.
> >>
> >> That said, apparently people have had a need to not be allowed to
> >> use some characters, but it's not clear why:
> >> https://bugs.launchpad.net/nova/+bug/977187 So I guess if anyone
> >> knows any reason why these printable characters should not be
> >> joined in holy resource naming, speak now or forever hold your
> >> peace.
> > 
> > I also could not find the reason of current restriction on the bug
> > report, and I'd like to know it as the history.
> > On v2 API(not v2.1), each resource name contains the following
> > restriction for its name:
> > 
> >   Resource  | Length  | Pattern
> >  ---+-+--
> >   aggregate | 1-255   | nothing
> >   backup| nothing | nothing
> >   flavor| 1-255   | '^[a-zA-Z0-9. _-]*[a-zA-Z0-9_-]+
> > | |   [a-zA-Z0-9. _-]*$'
> >   keypair   | 1-255   | '^[a-zA-Z0-9 _-]+$'
> >   server| 1-255   | nothing
> >   cell  | 1-255   | don't contain "." and "!"
> > 
> > On v2.1 API, we have applied the same restriction rule[1] for whole
> > resource names for API consistency, so maybe we need to consider
> > this topic for whole names.
> > 
> > [1]:
> > https://github.com/openstack/nova/blob/master/nova/api/validation/parameter_types.py#L44
> 
> Honestly, I bet this had to do with how the database used to be set
> up.
> 

So it turns out that utf8 support in MySQL does not support UTF-8 4 byte
multibyte characters (only 1-3 bytes). For example if you do a create
image call with an image name to glance with a 4 byte multibyte
character in the name it will 500. I'd guess we do something
similar in places with the Nova API where we have inadequate input
validation. If you want 4 byte support you need to use utf8mb4 instead.

I don't know if postgresql has any restrictions (I don't think it
does) or if db2 does too. But I don't think we can/should make it a
complete free for all. It should at most be what most databases support.

I think its a big enough change that this late in the cycle we should
push it off to Kilo. It's always much easier to loosen input validation
than tighten it (or have to have an "oops" revert on an officially
released Nova). Perhaps some tests to verify that everything we allow
past the input validation checks we can actually store.


> That being said, i'm pro letting names be 'utf8'. The id fields that
> are strings (like flavor_id) I think we should keep constrained, as
> we do actually do joins on them. (And as jay said, the current utf8
> schema is actually highly inefficient and bloaty.)

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Nova API meeting

[Christopher Yeoh]

Hi,

Just a reminder that the weekly Nova API meeting is being held tomorrow
Friday UTC . 

We encourage cloud operators and those who use the REST API such as
SDK developers and others who and are interested in the future of the
API to participate.

In other timezones the meeting is at:

EST 20:00 (Thu)
Japan 09:00 (Fri)
China 08:00 (Fri)
ACDT 9:30 (Fri)

The proposed agenda and meeting details are here: 

https://wiki.openstack.org/wiki/Meetings/NovaAPI

Please feel free to add items to the agenda. 

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] are we going to remove the novaclient v3 shell or what?

[Christopher Yeoh]

On Wed, 17 Sep 2014 17:42:30 +
"Day, Phil"  wrote:

> I think in the hopefully not too distant future we'll be able to make
> the v1_1 client handle both V2 and V2.1 (who knows. Maybe we can even
> rename it v2) - and that's what we should do because it will prove if
> we have full compatibility or not.

We should definitely do that cleanup in kilo. Though with microversions
hard coding a version number into the directory structure might not
make sense.

> Right now the two things holding that back are the V3 -> V2.1 API
> migration hasn't yet got to the point where the URLs include the
> tenant ID, and the URL still includes the "v3" tag - so the v3 client
> it the only easy way to exercise the v2.1 API in devstack.   For
> example I needed to do this to test the server group quota changes
> worked in devstack via V2.1 (I don't trust just getting the unit
> tests to pass).To do that I had to get boot working and accepting
> hints, and add support for the limits API.   So in the short term it
> seemed worth pushing those kinds of things back in for now in case
> they are useful to others.   Its not a big overhead to fix the v3 API
> at the same time (its mostly copied or sub-classed code) until we can
> add v2.1 support via the v2 client.

So the only reason I can think for keeping the v2.1/v3 version of
novaclient is if we want to break the python novaclient backwards
compatibility to do some cleanups. AFAIK we don't have a policy around
python novaclient backwards compatibility and if its ever ok to break
it.

Chris

> 
> Phil
> 
> > -Original Message-
> > From: Matt Riedemann [mailto:mrie...@linux.vnet.ibm.com]
> > Sent: 17 September 2014 15:59
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: [openstack-dev] [nova] are we going to remove the
> > novaclient v3 shell or what?
> > 
> > This has come up a couple of times in IRC now but the people that
> > probably know the answer aren't available.
> > 
> > There are python-novaclient patches that are adding new CLIs to the
> > v2 (v1_1) and v3 shells, but now that we have the v2.1 API (v2 on
> > v3) why do we still have a v3 shell in the client?  Are there plans
> > to remove that?
> > 
> > I don't really care either way, but need to know for code reviews.
> > 
> > One example: [1]
> > 
> > [1] https://review.openstack.org/#/c/108942/
> > 
> > --
> > 
> > Thanks,
> > 
> > Matt Riedemann
> > 
> > 
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Nova API meeting

[Christopher Yeoh]

Hi,

Just a reminder that the weekly Nova API meeting is being held tomorrow
Friday UTC . 

We encourage cloud operators and those who use the REST API such as
SDK developers and others who and are interested in the future of the
API to participate. 

In other timezones the meeting is at:

EST 20:00 (Thu)
Japan 09:00 (Fri)
China 08:00 (Fri)
ACDT 9:30 (Fri)

The proposed agenda and meeting details are here: 

https://wiki.openstack.org/wiki/Meetings/NovaAPI

Please feel free to add items to the agenda. 

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] FFE server-group-quotas

[Christopher Yeoh]

I'm willing to sponsor this


Chris
—
Sent from Mailbox

On Fri, Sep 5, 2014 at 10:29 PM, Day, Phil  wrote:

> Hi,
> I'd like to ask for a FFE for the 3 patchsets that implement quotas for 
> server groups.
> Server groups (which landed in Icehouse) provides a really useful 
> anti-affinity filter for scheduling that a lot of customers woudl like to 
> use, but without some form of quota control to limit the amount of 
> anti-affinity its impossible to enable it as a feature in a public cloud.
> The code itself is pretty simple - the number of files touched is a 
> side-effect of having three V2 APIs that report quota information and the 
> need to protect the change in V2 via yet another extension.
> https://review.openstack.org/#/c/104957/
> https://review.openstack.org/#/c/116073/
> https://review.openstack.org/#/c/116079/
> Phil
>> -Original Message-
>> From: Sahid Orentino Ferdjaoui [mailto:sahid.ferdja...@redhat.com]
>> Sent: 04 September 2014 13:42
>> To: openstack-dev@lists.openstack.org
>> Subject: [openstack-dev] [nova] FFE request serial-ports
>> 
>> Hello,
>> 
>> I would like to request a FFE for 4 changesets to complete the blueprint
>> serial-ports.
>> 
>> Topic on gerrit:
>> 
>> https://review.openstack.org/#/q/status:open+project:openstack/nova+br
>> anch:master+topic:bp/serial-ports,n,z
>> 
>> Blueprint on launchpad.net:
>>   https://blueprints.launchpad.net/nova/+spec/serial-ports
>> 
>> They have already been approved but didn't get enough time to be merged
>> by the gate.
>> 
>> Sponsored by:
>> Daniel Berrange
>> Nikola Dipanov
>> 
>> s.
>> 
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers

[Christopher Yeoh]

On Thu, 4 Sep 2014 12:57:57 -0700
Joe Gordon  wrote:

> 
> Overall I do think we need to re-think how the review burden is
> distributed. That being said, this is a nice proposal but I am not
> sure if it moves the review burden around enough or is the right
> approach. Do you have any rough numbers on what percent of the review
> burden goes to virt drivers today (how ever you want to define that
> statement, number of merged patches, man hours, lines of code, number
> of reviews  etc.). If for example today the nova review team spends
> 10% of there review time on virt drivers then I don't think this
> proposal will have a significant impact on the review backlog (for
> nova-common).

Even if it doesn't have a huge impact on the review backlog for
nova-common (I think it should at least help a bit) it does have the
potential to make life much easier for the virt driver developers. 

I think my main concern is around testing - as soon as we have multiple
repositories involved I think debugging of test failures
(especially races) tends to get more complicated and we have fewer
people who are familiar enough with the two code bases. 

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers

[Christopher Yeoh]

On Thu, 4 Sep 2014 11:24:29 +0100
"Daniel P. Berrange"  wrote:
> 
>  - A fairly significant amount of nova code would need to be
>considered semi-stable API. Certainly everything under nova/virt
>and any object which is passed in/out of the virt driver API.
>Changes to such APIs would have to be done in a backwards
>compatible manner, since it is no longer possible to lock-step
>change all the virt driver impls. In some ways I think this would
>be a good thing as it will encourage people to put more thought
>into the long term maintainability of nova internal code instead
>of relying on being able to rip it apart later, at will.
> 
>  - The nova/virt/driver.py class would need to be much better
>specified. All parameters / return values which are opaque dicts
>must be replaced with objects + attributes. Completion of the
>objectification work is mandatory, so there is cleaner separation
>between virt driver impls & the rest of Nova.

I think for this to work well with multiple repositories and drivers
having different priorities over implementing changes in the API it
would not just need to be semi-stable, but stable with versioning built
in from the start to allow for backwards incompatible changes. And
the interface would have to be very well documented including things
such as what exceptions are allowed to be raised through the API.
Hopefully this would be enforced through code as well. But as long as
driver maintainers are willing to commit to this extra overhead I can
see it working. 

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][FFE] v3-api-schema

[Christopher Yeoh]

On Thu, 4 Sep 2014 22:30:51 +0900
"Ken'ichi Ohmichi"  wrote:

> Hi
> 
> I'd like to request FFE for patches of v3-api-schema.
> The list  is the following:
> 
> https://review.openstack.org/#/c/67428/
> https://review.openstack.org/#/c/103437/
> https://review.openstack.org/#/c/103436/
> https://review.openstack.org/#/c/66783/
> 
> The one of them has already approved, but it stops merging with
> temporary -2. The other ones have gotten one +2 on each PS.
> This work will make v2.1 API strong.

I'm happy to sponsor this. This only affects the v2.1 API so is of low
risk to the rest of Nova

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] [FFE] alternative request for v2-on-v3-api

[Christopher Yeoh]

On Thu, 4 Sep 2014 23:08:09 +0900
"Ken'ichi Ohmichi"  wrote:

> Hi
> 
> I'd like to request FFE for v2.1 API patches.
> 
> This request is different from Christopher's one.
> His request is for the approved patches, but this is
> for some patches which are not approved yet.
> 
> https://review.openstack.org/#/c/113169/ : flavor-manage API
> https://review.openstack.org/#/c/114979/ : quota-sets API
> https://review.openstack.org/#/c/115197/ : security_groups API
> 
> I think these API are used in many cases and important, so I'd like
> to test v2.1 API with them together on RC phase.
> Two of them have gotten one +2 on each PS and the other one
> have gotten one +1.
> 

I'm happy to sponsor these extra changesets - I've reviewed them all
previously. Risk to the rest of Nova is very low.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] FFE request v2-on-v3-api

[Christopher Yeoh]

Hi,

I'd like to request a FFE for 4 changesets from the v2-on-v3-api
blueprint:

https://review.openstack.org/#/c/113814/
https://review.openstack.org/#/c/115515/
https://review.openstack.org/#/c/115576/
https://review.openstack.org/#/c/11/

They have all already been approved and were in the gate for a while
but just didn't quite make it through in time. So they shouldn't put any
load on reviewers.

Sponsoring cores:
Kenichi Ohmichi
John Garbutt
Me

Regards,

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] Lack of consistency in returning response from tempest clients

[Christopher Yeoh]

On Fri, 29 Aug 2014 11:13:39 -0400
David Kranz  wrote:

> On 08/29/2014 10:56 AM, Sean Dague wrote:
> > On 08/29/2014 10:19 AM, David Kranz wrote:
> >> While reviewing patches for moving response checking to the
> >> clients, I noticed that there are places where client methods do
> >> not return any value. This is usually, but not always, a delete
> >> method. IMO, every rest client method should return at least the
> >> response. Some services return just the response for delete
> >> methods and others return (resp, body). Does any one object to
> >> cleaning this up by just making all client methods return resp,
> >> body? This is mostly a change to the clients. There were only a
> >> few places where a non-delete  method was returning just a body
> >> that was used in test code.
> > Yair and I were discussing this yesterday. As the response
> > correctness checking is happening deeper in the code (and you are
> > seeing more and more people assigning the response object to _ ) my
> > feeling is Tempest clients should probably return a body obj that's
> > basically.
> >
> > class ResponseBody(dict):
> >  def __init__(self, body={}, resp=None):
> >  self.update(body)
> > self.resp = resp
> >
> > Then all the clients would have single return values, the body
> > would be the default thing you were accessing (which is usually
> > what you want), and the response object is accessible if needed to
> > examine headers.
> >
> > -Sean
> >
> Heh. I agree with that and it is along a similar line to what I
> proposed here https://review.openstack.org/#/c/106916/ but using a
> dict rather than an attribute dict. I did not propose this since it
> is such a big change. All the test code would have to be changed to
> remove the resp or _ that is now receiving the response. But I think
> we should do this before the client code is moved to tempest-lib.

+1. this would be a nice cleanup.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Nova API meeting

[Christopher Yeoh]

Hi,

Just a reminder that the weekly Nova API meeting is being held tomorrow
Friday UTC . 

We encourage cloud operators and those who use the REST API such as
SDK developers and others who and are interested in the future of the
API to participate. 

In other timezones the meeting is at:

EST 20:00 (Thu)
Japan 09:00 (Fri)
China 08:00 (Fri)
ACDT 9:30 (Fri)

The proposed agenda and meeting details are here: 

https://wiki.openstack.org/wiki/Meetings/NovaAPI

Please feel free to add items to the agenda. 

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Server Group API: add 'action' to authorizer?

[Christopher Yeoh]

On Sat, 23 Aug 2014 03:56:27 -0500
Joe Cropper  wrote:

> Hi Folks,
> 
> Would anyone be opposed to adding the 'action' checking to the v2/v3
> authorizers?  This would allow administrators more fine-grained
> control over  who can read vs. create/update/delete server groups.
> 
> Thoughts?
> 
> If folks are supportive, I'd be happy to add this... but not sure if
> we'd treat this as a 'bug' or whether there is a blueprint under which
> this could be done?

Long term we want to have a separate authorizer for every method. Alex
had a nova-spec  proposed for this but it unfortunately did not make
Juno

https://review.openstack.org/#/c/92326/

Also since the feature proposal deadline has passed it'll have to wait
till Kilo.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev