Re: [openstack-dev] [Neutron] Neutron extenstions
Hi Doug, On Thu, 19 Mar 2015 10:01:54 -0600 Doug Wiegley doug...@parksidesoftware.com wrote: Hi Gary, First I’m seeing these, but I don’t see that they’re required on input, unless I’m mis-reading those reviews. Additional of new output fields to a json object, or adding optional inputs, is not generally considered to be backwards incompatible behavior in an API. Does OpenStack have a stricter standard on that? So speaking from a Nova (and more general) API point of view where we're probably a bit longer along the extensions/microversioning path, I think even with backwards compatibility it is still an issue. For example, without some sort of co-operative and strong input validation are you going to detect when someone sends a new parameter that is simply mispelled and all the other extensions assume that someone will look after it and return an appropriate rather than let it just through. Over time we've found a few examples even in our api request samples typos (which feeds into the docs) which are not detected just because of this situation. Chris Thanks, doug On Mar 19, 2015, at 6:37 AM, Gary Kotton gkot...@vmware.com wrote: Hi, Changed the subject so that it may draw a little attention. There were 2 patches approved that kind of break the API (in my humble opinion): https://review.openstack.org/#/c/154921 https://review.openstack.org/#/c/154921/ and https://review.openstack.org/#/c/158420 https://review.openstack.org/#/c/158420/ In both of these two new fields were added to the base attributes – mtu and vlan_transparency Reverts for them are: https://review.openstack.org/165801 https://review.openstack.org/165801 (mtu) and https://review.openstack.org/165776 https://review.openstack.org/165776 (vlan transparency). In my opinion these should be added as separate extensions. Thanks Gary From: Gary Kotton gkot...@vmware.com mailto:gkot...@vmware.com Reply-To: OpenStack List openstack-dev@lists.openstack.org mailto:openstack-dev@lists.openstack.org Date: Thursday, March 19, 2015 at 2:32 PM To: OpenStack List openstack-dev@lists.openstack.org mailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Neutron] VLAN transparency support Hi, This patch has the same addition too - https://review.openstack.org/#/c/154921 https://review.openstack.org/#/c/154921/. We should also revert that one. Thanks Gary From: Gary Kotton gkot...@vmware.com mailto:gkot...@vmware.com Reply-To: OpenStack List openstack-dev@lists.openstack.org mailto:openstack-dev@lists.openstack.org Date: Thursday, March 19, 2015 at 1:14 PM To: OpenStack List openstack-dev@lists.openstack.org mailto:openstack-dev@lists.openstack.org Subject: [openstack-dev] [Neutron] VLAN transparency support Hi, It appears that https://review.openstack.org/#/c/158420 https://review.openstack.org/#/c/158420/ update the base attributes for the networks. Is there any reason why this was not added as a separate extension like all others. I do not think that this is the correct way to go and we should do this as all other extensions have been maintained. I have posted a revert (https://review.openstack.org/#/c/165776 https://review.openstack.org/#/c/165776/) – please feel free to knack if it is invalid. Thanks Gary __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Tue, Mar 24, 2015 at 5:45 AM, Jay Pipes jaypi...@gmail.com wrote:
On Tue, Mar 10, 2015 at 09:52:14PM +1030, Christopher Yeoh wrote:
On Mon, 09 Mar 2015 16:14:21 -0400
Sean Dague s...@dague.net wrote:
On 03/09/2015 03:37 PM, Jay Pipes wrote:
On 03/08/2015 08:10 AM, Alex Xu wrote:
Thanks for Jay point this out! If we have agreement on this and
document it, that will be great for guiding developer how to add
new API.
I know we didn't want extension for API. But I think we still
need modularity. I don't think we should put everything in a single
file, that file will become huge in the future and hard to
maintenance.
I don't think everything should be in a single file either. In fact,
I've never advocated for that.
We can make the 'extension' not configurable. Replace 'extension'
with another name, deprecate the extension info api int he
future But that is not mean we should put everything in a file.
I didn't say that in my email. I'm not sure where you got the
impression I want to put everything in one file?
For modularity, we need define what should be in a separated
module(it is extension now.) There are three cases:
1. Add new resource
This is totally worth to put in a separated module.
Agreed.
2. Add new sub-resource
like server-tags, I prefer to put in a separated module, I
don't think put another 100 lines code in the servers.py is good
choice.
Agreed, which is exactly what I said in my email:
Similarly, new microversion API functionality should live in a
module, as a top-level (or subcollection) Controller in
/nova/api/openstack/compute/, and should not be in the
/nova/api/openstack/compute/plugins/ directory. Why? Because it's
not a plugin.
Ok so I'm getting confused about what we're disagreeing about then.
I wasn't disagreeing with you. I was disagreeing with Alex Xu :) Perhaps
your mail client was confusing you?
However, the first microversion change
https://review.openstack.org/#/c/140313/32 is one case where we didn't
need to create a new extension, relying only on microversions to add a
new parameter to the response, whereas server tags does add a new
conroller (os-server-tags) which is non trivia so I think it does. Do
we disagree about that?
The server tags patch does add more functionality to the API, for sure.
But, as I've written about in this thread, there's no reason at all that
it needed to use the extension framework. It could better have been
written as a simple Controller object in a new file in
/nova/api/openstack/compute/server_tags.py, with no use of the
extension mechanism whatsoever.
btw in a situation now where (I think) we are saying we are not going
to do any work for 3rd parties to add their own API plugins and have a
well planned API we don't need the prefixes as we don't need the prefix
on parameter names as there won't be name clashes without us realising
during testing. And we certainly don't need the os- prefix is the
plugin alias, but we do neeed them unique across the API I believe
because of the way we store information about them.
Right. There won't be any more name clashes if we don't allow API
extensions any more. Which is what I'm asking for.
3. extend attributes and methods for a existed resource
like add new attributes for servers, we can choice one of
existed module to put it in. Just like this patch
https://review.openstack.org/#/c/155853/
But for servers-tags, it's sub-resource, we can put it in
its-own module.
Agreed, and that's what I put in my email.
If we didn't want to support extension right now, we can begin
from not show servers-tags in extension info API first. That means
extension info is freeze now. We deprecated the extension info api
in later version.
It can be surpressed from /extensions by adding it to the suppress list
in the extensions code. Probably a good idea to stop v2.1+microversion
code REST API users not accidentally relying on it.
I don't want it suppressed. I want the use of API extensions and the
extension framework(s) to be completely dropped for all future
API-affecting work.
I don't understand what you're saying here. Could you elaborate?
What I am asking for is for new functionality (like the server-tags
subcollection resource), just add a new module called
/nova/api/openstack/compute/server_tags.py, create a Controller
object in that file with the new server tags resource, and don't
use any of the API extensions framework whatsoever.
In addition to that, for the changes to the main GET
/servers/{server_id} resource, use microversions to decorate the
/nova/api/openstack/compute/servers.py.Controller.show() method for
2.4 and add a tags key to the dict (note: NOT a
os-server
Re: [openstack-dev] [nova] what is our shipped paste.ini going to be for Kilo
On Mon, 16 Mar 2015 11:22:12 -0400 Russell Bryant rbry...@redhat.com wrote: On 03/16/2015 11:08 AM, John Garbutt wrote: While its not under Nova's control, I think we need to consider the keystone catalog here. It feels nice to have an explicit entry for v2.1, that people start using once the client is updated to support microversions. DefCore would eventually check for the presence of that entry. Eventually, the v2 entry would be served by the v2.1 code, once we are happy its not going to break too many folks, at least including all the major SDKs. I hope thats during Liberty. I'd really rather get away from any explicit entries in the keystone service catalog for API versions. It seems kind of broken to me. I think the service catalog should only have enough to point you to the APIs location. Discovering and choosing versions should be done by the client and not embedded in the service catalog. I agree, though the historical issue of why we have multiple keystone entries for the Nova API is because the first one pointed to a specific version rather than a generic place where a client can do version discovery. We probably need to dig ourselves out of that hole first (which is doable I think in conjunction with JSON-HOME which is an L thing) Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] what is our shipped paste.ini going to be for Kilo
On Mon, 16 Mar 2015 10:15:50 -0400 Sean Dague s...@dague.net wrote: Our current top level shipped example paste.ini for Nova includes the following set of endpoint definitions: [composite:osapi_compute] use = call:nova.api.openstack.urlmap:urlmap_factory /: oscomputeversions /v1.1: openstack_compute_api_v2 /v2: openstack_compute_api_v2 /v2.1: openstack_compute_api_v21 /v3: openstack_compute_api_v3 The real question I have is what should this look like in the Kilo release. And this has a couple of axes. Who uses our paste.ini? = paste.ini is an etc file, so we assume that during upgrade you'll be using your existing config. Changes to the default paste.ini will really only be effective in new deploys. So this should not impact existing users, but instead only new users. Cleaning up Cruft = Drop of /v3 --- v3 is no longer a supported thing. I think v3 in the paste.ini causes confusion. It also causes us to keep around untested / unsupported code. This seems kind of a no brainer. Drop of /v1.1 ? --- Only new deploys are really going to base off of our in tree config. I'm not convinced that we want to encourage people setting up new /v1.1 endpoint in tree. +1 to dropping this. Can we also drop novaclient support for v1.1 at some point. I too would like to know if anyone actually uses it I'm not convinced there is a deprecation path here because the ones I could imagine would involve people changing their paste.ini to include a deprecation checking piece of code. Honestly, I don't care strongly either way on this one. It's cruft, but not dangerous cruft (unlike v3). Nova v2 === This is where things get interesting. v2.1 is supposed to be equivalent to v2. The difference is moving the validation for datastructures from the database to the wsgi layer. The ways in which this doesn't react like the existing APIs should be basically not letting you create corrupt data models which will explode later in unexpected and hard to determine ways. The reality is objects validation has been sneaking this in all along anyway. The full monty approach --- [composite:osapi_compute] use = call:nova.api.openstack.urlmap:urlmap_factory /: oscomputeversions /v1.1: openstack_compute_api_v2 /v2: openstack_compute_api_v21 # starting in Kilo the v21 implementation replaces the v2 # implementation and is suggested that you use it as the default. If # this causes issues with your clients you can rollback to the # *frozen* v2 api by commenting out the above stanza and using the # following instead:: # /v2: openstack_compute_api_v2 # if rolling back to v2 fixes your issue please file a critical bug # at - https://bugs.launchpad.net/nova/+bugs This would make the v2 endpoint the v21 implementation for new deploys. It would also make it easy for people to flip back if they hit an edge condition we didn't notice. In functional testing we'd still test both v2 and v2.1 Tempest would move to v2.1 by default, and I think we should put an old v2 job on nova stable/kilo - master to help us keep track of regressions. The slow roll approach -- Ship the existing file (minus v3): [composite:osapi_compute] use = call:nova.api.openstack.urlmap:urlmap_factory /: oscomputeversions /v1.1: openstack_compute_api_v2 /v2: openstack_compute_api_v2 /v2.1: openstack_compute_api_v21 The advantages here is that out of the box stuff keeps working. The dilema here is it's not really clear that we'll get people poking at v2.1 because it will be out of the main path. The point of the microversioning was to get folks onto that train soon because it falls back to existing API. And once we are convinced we're good we can deprecate out the old implementation. Also, things like out of tree EC2 support requires v2.1, which is going to make deploys start relying on a /v2.1 endpoint that want EC2, so our options for grafting that back onto /v2 in the future are more limitted. Decision Time = Anyway, this is a decision we should make before freeze. The 'no decision' case gives us the slow roll. I think from an upstream perspective the full monty will probably serve us a little better. Especially with robust release notes that explain to people how to move their endpoints forward. -Sean __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] what is our shipped paste.ini going to be for Kilo
On Tue, 17 Mar 2015 15:56:27 +1300 Robert Collins robe...@robertcollins.net wrote: On 17 March 2015 at 14:27, Ken'ichi Ohmichi ken1ohmi...@gmail.com wrote: I am worried about SDKs making requests that have additional JSON attributes that were previously ignored by v2, but will be considered invalid by the v2.1 validation code. If we were to just strip out the extra items, rather than error out the request (when you don't specify a microversion), I would be less worried about the transition. Maybe that what we do? Nice point. That is a main difference in API behaviors between v2 and v2.1 APIs. If SDKs pass additional JSON attributes to Nova API now, developers need to fix/remove these attributes because that is a bug on SDKs side. These attributes are unused and meaningless, so some APIs of SDKs would contain problems if passing this kind of attributes. Sometime it was difficult to know what are available attributes before v2.1 API, so The full monty approach will clarify problems of SDKs and make SDKs' quality better. Thanks Ken Ohmichi Better at the cost of forcing all existing users to upgrade just to keep using code of their own that already worked. Not really 'better' IMO. Different surely. We could (should) add Warning: headers to inform about this, but breaking isn't healthy IMO. It'd be up to the operators, but there is always the option of simply editing the paste.ini file so /v2 is again the produced by the old v2 code. My main concern about v2 / v2.1 compatibility in practicce (rather than just passing the same tempest and uniteststs which does work) is lack of feedback. Probably don't exepct positive feedback in many cases but we're not really getting negative feedback much either. I really would appreciate people actually trying it more real world apps so we get a better idea of the compatibility in areas of the code that don't have good tempest coverage or have unitests which are incomplete. Regards, Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] how safe is it to change NoAuthMiddlewareBase?
On Sat, 28 Feb 2015 09:51:27 -0700 Jay Pipes jaypi...@gmail.com wrote: On 02/26/2015 04:27 AM, Sean Dague wrote: In trying to move the flavor manage negative tests out of Tempest and into the Nova functional tree, I ran into one set of tests which are permissions checking. Basically that a regular user isn't allowed to do certain things. In (nearly) all our tests we use auth_strategy=noauth which takes you to NoAuthMiddlewareBase instead of to keystone. That path makes you an admin regardless of what credentials you send in - https://github.com/openstack/nova/blob/master/nova/api/openstack/auth.py#L56-L59 What I'd like to do is to change this so that if you specify user_id='admin' then is_admin is set true, and it's not true otherwise. That has a bunch of test fall out, because up until this point most of the test users are things like 'fake', which would regress to non admin. About 25% of the api samples tests fail in such a change, so they would need to be fixed. Taking a step back... what exactly is the purpose of the API samples functional tests? If the purpose of these tests has anything to do with validating some policy thing, then I suppose it's worth changing the auth middleware to support non-adminness. But, Historically I think its been a couple of reasons - to generate api samples for documentation purposes - to do more thorough testing of Nova that previously should have gone into tempest (but everything is moving back now right?) but writing tempests tests has been seen as too hard a postponed until after the Nova change has merged and then the intent has been lost. I don't think the API samples test purpose has anything to do with that (I think the purpose of the API samples tests is fuzzy, at best, actually). So, I'd just leave them as-is and not change anything at all. If we're moving stuff over from tempest to nova we definitely need to keep tracking of what has been done and what we need to do. Eg we need to find out what state we're in first. Definitely have picked up lots of issues in the past with the functional tests that the unitests have missed. Chris Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] is it possible to microversion a static class method?
So ultimately I think this is a style issue rather than a technical one. I think there are situations where one way looks clearer than another the other way does. Sorry I can't get around to putting up a couple of examples, ATM but to be clear there is no difference in the end result (no different side effects etc) On Mon, Mar 16, 2015 at 12:21 PM, Alex Xu sou...@gmail.com wrote: 2015-03-16 9:48 GMT+08:00 Alex Xu sou...@gmail.com: 2015-03-13 19:10 GMT+08:00 Sean Dague s...@dague.net: On 03/13/2015 02:55 AM, Chris Friesen wrote: On 03/12/2015 12:13 PM, Sean Dague wrote: On 03/12/2015 02:03 PM, Chris Friesen wrote: Hi, I'm having an issue with microversions. The api_version() code has a comment saying This decorator MUST appear first (the outermost decorator) on an API method for it to work correctly I tried making a microversioned static class method like this: @wsgi.Controller.api_version(2.4) # noqa @staticmethod def _my_func(req, foo): and pycharm highlighted the api_version decorator and complained that This decorator will not receive a callable it may expect; the built-in decorator returns a special object. Is this a spurious warning from pycharm? The pep8 checks don't complain. If I don't make it static, then pycharm suggests that the method could be static. *API method* This is not intended for use by methods below the top controller level. If you want conditionals lower down in your call stack pull the request version out yourself and use that. Both the original spec and doc/source/devref/api_microversions.rst contain text talking about decorating a private method. The latter gives this example: @api_version(2.1, 2.4) def _version_specific_func(self, req, arg1): pass @api_version(min_version=2.5) #noqa def _version_specific_func(self, req, arg1): pass def show(self, req, id): common stuff self._version_specific_func(req, foo) common stuff It's entirely possible that such a private method might not need to reference self, and could therefore be static, so I think it's a valid question. That's a doc bug, we should change it. Actually it is not a bug. It's controversial point in the spec, but finally that was keep in the spec. http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/api-microversions.html The discussion at line 268 https://review.openstack.org/#/c/127127/7/specs/kilo/approved/api-microversions.rst Submit a patch for devref https://review.openstack.org/164555 Let see whether we can get agreement -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] need input on possible API change for bug #1420848
FWIW I think we need to consider that the API is completely froxen for the V2 API (so this freeze does not apply to v2.1 microversions) except under very serious circumstances and only very high priority bug fixes and only apply this to a suitable microversion bump. We really want to get rid of the V2 API code asap anyway. You can after all request a version number of a per method basis as long as you are talking to v2.1. So the only forced upgrade is the v2-v2.1 transition and those apis should be identical On Fri, Mar 13, 2015 at 12:02 AM, Christopher Yeoh cbky...@gmail.com wrote: On Thu, Mar 12, 2015 at 11:19 PM, Christopher Yeoh cbky...@gmail.com wrote: On Wed, 11 Mar 2015 09:32:11 -0600 Chris Friesen chris.frie...@windriver.com wrote: Hi, I'm working on bug #1420848 which addresses the issue that doing a service-disable followed by a service-enable against a down compute node will result in the compute node going up for a while, possibly causing delays to operations that try to schedule on that compute node. The proposed solution is to add a new reported_at field in the DB schema to track when the service calls _report_state(). The backend is straightforward, but I'm trying to figure out the best way to represent this via the REST API response. Currently we response includes an updated_at property, which maps directly to the auto-updated updated_at field in the database. Would it be acceptable to just put the reported_at value (if it exists) in the updated_at property? Logically the reported_at value is just a determination of when the service updated its own state, so an argument could be made that this shouldn't break anything. So i think this is the critical point here is this a backwards compatibly API change or not. Would reporing reported_at in updated_at cause *anyone* any pain. For this reason I think it has to go through as a nova spec (and if you think its not going to cause pain get some people from the mailing list +1 it as backwards API change because it always has been a bug. If that is the conculsion and you just reuse updated_at then the procedure would be: - Add it to v2 and no v2 extension required - Add it to v2.1 without an extension - No change required to in terms of microversions because it is lready in the base v2.1 code If you go the reported_at route the and there no changed to updated_at but the fix is consiered a new feature rather than a bug fix then I think we should seriously consider if it should be fixed in V2 at all because the v2 api is basically frozen and we can just add it as a microversion (don't even need to to support it in v2.1), just api microversion In which case the documents that Kevin pointed to should help - if you have any problems catch me on irc or on return email __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] need input on possible API change for bug #1420848
On Thu, Mar 12, 2015 at 11:19 PM, Christopher Yeoh cbky...@gmail.com wrote: On Wed, 11 Mar 2015 09:32:11 -0600 Chris Friesen chris.frie...@windriver.com wrote: Hi, I'm working on bug #1420848 which addresses the issue that doing a service-disable followed by a service-enable against a down compute node will result in the compute node going up for a while, possibly causing delays to operations that try to schedule on that compute node. The proposed solution is to add a new reported_at field in the DB schema to track when the service calls _report_state(). The backend is straightforward, but I'm trying to figure out the best way to represent this via the REST API response. Currently we response includes an updated_at property, which maps directly to the auto-updated updated_at field in the database. Would it be acceptable to just put the reported_at value (if it exists) in the updated_at property? Logically the reported_at value is just a determination of when the service updated its own state, so an argument could be made that this shouldn't break anything. So i think this is the critical point here is this a backwards compatibly API change or not. Would reporing reported_at in updated_at cause *anyone* any pain. For this reason I think it has to go through as a nova spec (and if you think its not going to cause pain get some people from the mailing list +1 it as backwards API change because it always has been a bug. If that is the conculsion and you just reuse updated_at then the procedure would be: - Add it to v2 and no v2 extension required - Add it to v2.1 without an extension - No change required to in terms of microversions because it is lready in the base v2.1 code If you go the reported_at route the and there no changed to updated_at but the fix is consiered a new feature rather than a bug fix then I think we should seriously consider if it should be fixed in V2 at all because the v2 api is basically frozen and we can just add it as a microversion (don't even need to to support it in v2.1), just api microversion In which case the documents that Kevin pointed to should help - if you have any problems catch me on irc or on return email Otherwise, by my reading of https://wiki.openstack.org/wiki/APIChangeGuidelines#Generally_Considered_OK it seems like if I wanted to add a new reported_at property I would need to do it via an API extension. Anyone have opinions? Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Mon, 09 Mar 2015 16:14:21 -0400
Sean Dague s...@dague.net wrote:
On 03/09/2015 03:37 PM, Jay Pipes wrote:
On 03/08/2015 08:10 AM, Alex Xu wrote:
Thanks for Jay point this out! If we have agreement on this and
document it, that will be great for guiding developer how to add
new API.
I know we didn't want extension for API. But I think we still
need modularity. I don't think we should put everything in a single
file, that file will become huge in the future and hard to
maintenance.
I don't think everything should be in a single file either. In fact,
I've never advocated for that.
We can make the 'extension' not configurable. Replace 'extension'
with another name, deprecate the extension info api int he
future But that is not mean we should put everything in a file.
I didn't say that in my email. I'm not sure where you got the
impression I want to put everything in one file?
For modularity, we need define what should be in a separated
module(it is extension now.) There are three cases:
1. Add new resource
This is totally worth to put in a separated module.
Agreed.
2. Add new sub-resource
like server-tags, I prefer to put in a separated module, I
don't think put another 100 lines code in the servers.py is good
choice.
Agreed, which is exactly what I said in my email:
Similarly, new microversion API functionality should live in a
module, as a top-level (or subcollection) Controller in
/nova/api/openstack/compute/, and should not be in the
/nova/api/openstack/compute/plugins/ directory. Why? Because it's
not a plugin.
Ok so I'm getting confused about what we're disagreeing about then.
However, the first microversion change
https://review.openstack.org/#/c/140313/32 is one case where we didn't
need to create a new extension, relying only on microversions to add a
new parameter to the response, whereas server tags does add a new
conroller (os-server-tags) which is non trivia so I think it does. Do
we disagree about that?
btw in a situation now where (I think) we are saying we are not going
to do any work for 3rd parties to add their own API plugins and have a
well planned API we don't need the prefixes as we don't need the prefix
on parameter names as there won't be name clashes without us realising
during testing. And we certainly don't need the os- prefix is the
plugin alias, but we do neeed them unique across the API I believe
because of the way we store information about them.
3. extend attributes and methods for a existed resource
like add new attributes for servers, we can choice one of
existed module to put it in. Just like this patch
https://review.openstack.org/#/c/155853/
But for servers-tags, it's sub-resource, we can put it in
its-own module.
Agreed, and that's what I put in my email.
If we didn't want to support extension right now, we can begin
from not show servers-tags in extension info API first. That means
extension info is freeze now. We deprecated the extension info api
in later version.
It can be surpressed from /extensions by adding it to the suppress list
in the extensions code. Probably a good idea to stop v2.1+microversion
code REST API users not accidentally relying on it.
I don't understand what you're saying here. Could you elaborate?
What I am asking for is for new functionality (like the server-tags
subcollection resource), just add a new module called
/nova/api/openstack/compute/server_tags.py, create a Controller
object in that file with the new server tags resource, and don't
use any of the API extensions framework whatsoever.
In addition to that, for the changes to the main GET
/servers/{server_id} resource, use microversions to decorate the
/nova/api/openstack/compute/servers.py.Controller.show() method for
2.4 and add a tags key to the dict (note: NOT a
os-server-tags:tags key) returned by GET /servers/{server_id}. No
use of API extensions needed.
So I that's doabe but I think if we compared to the two wsgi.extends
is cleaner and less code and we have to have the separate module file
anyway for the controller. Can discuss this more later
Incidentally as has been mentioned before we need new names for the
API and where the files need to live needs cleaning up. For example v3
and v2.1 needs to be worked out of the directory paths.This clenaup not
on involves but directly affects the all the related unit and
functional tests. Nor should we have contrib or should v2 live directly
in in nova/api/openstack/compute. But we also need a name for v2.1
microversions and should spend some time on that (does api modules work
for anyone?)
I think this dicussion indicates we should start with a bit of planning
first rather than just jump in and start shuffling this around now.
Work out what we want the final thing to look like so we can see what
the dependencies look like and minimise the number of times
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt j...@johngarbutt.com wrote: Hi, I think I agree with Jay here, but let me explain... On 8 March 2015 at 12:10, Alex Xu sou...@gmail.com wrote: Thanks for Jay point this out! If we have agreement on this and document it, that will be great for guiding developer how to add new API. +1 Please could you submit a dev ref for this? We can argue on the review, a bit like this one: https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst For modularity, we need define what should be in a separated module(it is extension now.) There are three cases: 1. Add new resource This is totally worth to put in a separated module. +1 2. Add new sub-resource like server-tags, I prefer to put in a separated module, I don't think put another 100 lines code in the servers.py is good choice. -1 I hate the idea of show instance extension code for version 2.4 living separately to the rest of the instance show logic, when it really doesn't have to. It feels too heavyweight in its current form. If the only thing server-tags did was to add a parameter then we wouldn't need a new extension, but its not, it adds another resource with associated actions Maybe we need a more modular way of expressing the extension within the same file? I think servers.py is simply to big. Its much harder to read and debug than any other plugin just because of its size - or maybe I just need a 50 monitor :) I'd rather ensure functionality common server-tags and the API is kept together rather than spread through servers.py 3. extend attributes and methods for a existed resource like add new attributes for servers, we can choice one of existed module to put it in. Just like this patch https://review.openstack.org/#/c/155853/ +1 I wish it was easier to read, but I hope thats fixable long term. 2015-03-08 8:31 GMT+08:00 Jay Pipes jaypi...@gmail.com: Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. As above, a new resource probably should get a new plugins/v3 module right? It feels (at worst) borderline in the os-server-tags case, due to the extra actions. What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. Everything is a plugin in v3, no more distinction between core vs plugin. It needs renaming really. It should look just like servers, I guess, which is a top level item: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/plugins/v3/servers.py Why are we continuing to use these awkward, messy, and cumbersome API extensions? We certainly should never be forced to add an extension to advertise new functionality anymore. Its a big reason why I want to see the API micro-versions succeed. Yep, there is I think no reason except to support /extensions for now and I don't really think its worth having two entry points, one for modules which will appear in /extensions and one for modules which won't appear in /extensioins. The overhead is low. We should warn v2.1+ users to ignore /extensions unless they are legacy v2 api users and they should remove their use of it anyway as soon as they get off v2.1. They key to dumping it all is when people tell us v2.1 really is behaving just like v2 so we can remove the old v2 code and then later have a microversion that doesn't support /extensions. I hope all the json-home stuff is in by then :-) Please, I am begging the Nova core team. Let us stop this madness. No more API extensions. Lets try get something agreed in devref, so we are ready to go when Liberty opens. It would be nice to look at ways to fold back the existing extensions into the main code. I know there are v2.0 compatibility issues there, but I think/hope thats mostly cosmetic at this point. Yea we already did a lot of that in v3 and had to separate some of them out again for v2.1 (argh!). Others we have just faked (eg you load module X you get module Y for free which doesn't really exist anymore - but only for those that we were very sure that the extension only existed to notify users that some functionality was present. Thanks, John
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt j...@johngarbutt.com wrote: +1 Please could you submit a dev ref for this? We can argue on the review, a bit like this one: https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst I think it'd also be a good idea to add a testcase (use test_plugins directory where you can define your own controller which is never plubished) for each example so they don't get out of date Regards, Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Mon, Mar 9, 2015 at 9:35 PM, Sean Dague s...@dague.net wrote: On 03/07/2015 07:31 PM, Jay Pipes wrote: Hi Stackers, Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. Sergey Nikitin is currently in the process of code review for the final patch that adds server instance tagging to the Nova API: https://review.openstack.org/#/c/128940 Unfortunately, for some reason I really don't understand, Sergey is being required to create an API extension called os-server-tags in order to add the server tag functionality to the API. The patch implements the 2.4 Nova API microversion, though, as you can see from this part of the patch: https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369 Because we're using an API extension for this new server tags functionality, we are instead having the extension extend the server dictionary with an os-server-tags:tags key containing the list of string tags. This is ugly and pointless. We don't need to use API extensions any more for this stuff. A client knows that server tags are supported by the 2.4 API microversion. If the client requests the 2.4+ API, then we should just include the tags attribute in the server dictionary. Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. Why are we continuing to use these awkward, messy, and cumbersome API extensions? Please, I am begging the Nova core team. Let us stop this madness. No more API extensions. Agreed, the current extensions list exists to explain the base v2 functionality. I think we should consider that frozen and deprecated as of v2.1 as we have a better way to express features. -Sean So I think we can a microversion ASAP to remove support for /extensions. Obviously we'll need to keep the actual code to support v2.1 for quite a while though. I think we still want some fields in the controller like we do because we want to automate JSON-HOME/Schema stuff (maybe not sure) -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
Hi, Apologies for the slow reply, long weekend because of a public holiday over here. I'm probably going to end up repeating part of what Alex has mentioned as well. So the first thing I think we want to distinguish between plugins being a REST API user or operator concept and it being a tool developers use as a framework to support the Nova REST API. As I've mentioned before I've no problem with the feature set of the API being fixed (per microversion) across all Nova deployments. Get back to me when we have consensus on that and its trivial to implement and we'll no longer have the concept of core and extension/plugin. But plugin like implementations using Stevedore as a tool for developers to keep good modularity has proven to be very useful to keep complexity level lower and interactions between modules much clearer. servers.py is an example of this where in v2 I think we have/had the most complex method and even with all the fix up work which has been done on it it is still very complicated to understand. On Sun, Mar 8, 2015 at 11:01 AM, Jay Pipes jaypi...@gmail.com wrote: Hi Stackers, Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. Sergey Nikitin is currently in the process of code review for the final patch that adds server instance tagging to the Nova API: https://review.openstack.org/#/c/128940 Unfortunately, for some reason I really don't understand, Sergey is being required to create an API extension called os-server-tags in order to add the server tag functionality to the API. The patch implements the 2.4 Nova API microversion, though, as you can see from this part of the patch: https://review.openstack.org/#/c/128940/43/nova/api/ openstack/compute/plugins/v3/server_tags.py What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? Actually I think it does more than just add extra reponse information: - it adds extra tags parameter to show - it doesn't add it to index, but it probably should add the response information to detail to be consistent with the rest of the API - It adds a new resource /servers/server_id/tags - with create, delete and delete all supported. I don't think that these belong in servers.py https://github.com/openstack/nova/blob/master/nova/api/ openstack/compute/servers.py#L369 Because we're using an API extension for this new server tags functionality, we are instead having the extension extend the server dictionary with an os-server-tags:tags key containing the list of string tags. This is ugly and pointless. We don't need to use API extensions any more for this stuff. So we had a prefix rule originally in V2 to allow for extensions and guarantee no name clashes. I'd be happy removing this requirement, even removing old ones as long as we have consensus. A client knows that server tags are supported by the 2.4 API microversion. If the client requests the 2.4+ API, then we should just include the tags attribute in the server dictionary. Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. So I don't see how that changes whether we're using plugins (from a user point of view) or not. The good news for you is that there is fixing the shambles of a directory structure for the api is on the list of things to do, it just wasn't a high prioirty things for us in Kilo, get v2.1 and microversions out. For example, we have v3 in the directory path as well for historical reasons and we also have a contrib directory in compute and none of those are really contrib now either. Now the nova/api/openstack/compute/ directory where you want to put all the v2 microversions code is currently full of v2 core code already. It just makes more sense to me to wait unti the old v2 core code can be removed because the v2.1 api is considered equivalent and then move the v2.1 microversions code into its final place , rather than a shuffle now to move the old v2 code (along with all the changes need to the unittests) and then just have to delete it again not much longer. Why are we continuing to use these awkward, messy, and cumbersome API extensions? Please, I am begging the Nova core team. Let us stop this madness. No more API extensions. It is still not clear to me exactly what you mean by use of an extension. None of us ours are built for real runtime loading/unloading of
Re: [openstack-dev] [nova] what's the merge plan for current proposed microversions?
On Wed, Mar 4, 2015 at 9:51 PM, Alexandre Levine alev...@cloudscaling.com wrote: Christopher, Does this So the plan for assignment of microversion api numbers is the same as what we currently do for db migration changes - take the next one knowing that you may need to rebase if someone else merges before you mean that I should put 2.2 in my review for now instead of 2.4? I don't think that'd be worth it because in this case as its the first microversion we've already decided to merge https://review.openstack.org/140313 first and you'll need to rebase to at least 2.3. I think the reason I recommended 2.4 to you was that https://review.openstack.org/128940 was the other patch identified as a possible good candidate for being the first microversion but in the end wasn't selected so it ended up with 2.3. From a quick look at 128940 I think it might have spec approval issues thoughso if your patch is ready when 140313 merges you might end up with 2.3. Other suggestion would be to pack several simultaneously incoming changes into one microversion. Maybe spawn them once a week, or once a couple of weeks, or even with longer period, depending on the amount of incoming changes. For example, wouldn't it be convenient for clients to acquire all of the accumulated changes in one microversion (2.2 as I understand) without needing to understand which one came with what number? To clarify, I'm suggesting to pass reviews for all of the hanging API changes against 2.2 version. So I think its probably better to keep the number of changes small per microversion. Though I have also suggested in the past that very minor changes in the same such as formatting etc be fixed if we're making api chnges in the same area anyway and clients will be forced to modify what they do regardless. However bundling a lot of api changes in one microversion will for CD we'd need them to be enabled all at once meaning we'd either need to merge them into one patch or have an additional patch which just enables say 2.2 once all the dependent patches are merged. This increases the probability that one delayed patch will delay a bunch of others whereas now whoever is ready the first will merge first.. It someone has experience from db migrations that they think would work well, please let us know! Regards, Chris Best regards, Alex Levine On 3/4/15 11:44 AM, Christopher Yeoh wrote: On Tue, 03 Mar 2015 10:28:34 -0500 Sean Dague s...@dague.net wrote: On 03/03/2015 10:24 AM, Claudiu Belu wrote: Hello. I've talked with Christopher Yeoh yesterday and I've asked him about the microversions and when will they be able to merge. He said that for now, this commit had to get in before any other microversions: https://review.openstack.org/#/c/159767/ He also said that he'll double check everything, and if everything is fine, the first microversions should be getting in soon after. Best regards, Claudiu Belu I just merged that one this morning, so hopefully we can dislodge. So just before we merged the the keypairs microversion change someone enabled response schema tests which showed some further problems. They have now all been fixed but https://review.openstack.org/#/c/161112/ which needs just one more +2. After that the first api change which uses microversions https://review.openstack.org/#/c/140313/ can merge (has 3 +2's just needs the v2.1 fix first. From: Alexandre Levine [alev...@cloudscaling.com] Sent: Tuesday, March 03, 2015 4:22 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [nova] what's the merge plan for current proposed microversions? Bump. I'd really appreciate some answers to the question Sean asked. I still have the 2.4 in my review (the very one Sean mentioned) but it seems that it might not be the case. Best regards, Alex Levine On 3/2/15 2:30 PM, Sean Dague wrote: This change for the additional attributes for ec2 looks like it's basically ready to go, except it has the wrong microversion on it (as they anticipated the other changes landing ahead of them) - https://review.openstack.org/#/c/155853 What's the plan for merging the outstanding microversions? I believe we're all conceptually approved on all them, and it's an important part of actually moving forward on the new API. It seems like we're in a bit of a holding pattern on all of them right now, and I'd like to make sure we start merging them this week so that they have breathing space before the freeze. So the plan for assignment of microversion api numbers is the same as what we currently do for db migration changes - take the next one knowing that you may need to rebase if someone else merges before you. Other suggestions welcome but will have to follow the requirement that they always merge in version order. -Sean
Re: [openstack-dev] [nova] what's the merge plan for current proposed microversions?
On Tue, 03 Mar 2015 10:28:34 -0500 Sean Dague s...@dague.net wrote: On 03/03/2015 10:24 AM, Claudiu Belu wrote: Hello. I've talked with Christopher Yeoh yesterday and I've asked him about the microversions and when will they be able to merge. He said that for now, this commit had to get in before any other microversions: https://review.openstack.org/#/c/159767/ He also said that he'll double check everything, and if everything is fine, the first microversions should be getting in soon after. Best regards, Claudiu Belu I just merged that one this morning, so hopefully we can dislodge. So just before we merged the the keypairs microversion change someone enabled response schema tests which showed some further problems. They have now all been fixed but https://review.openstack.org/#/c/161112/ which needs just one more +2. After that the first api change which uses microversions https://review.openstack.org/#/c/140313/ can merge (has 3 +2's just needs the v2.1 fix first. From: Alexandre Levine [alev...@cloudscaling.com] Sent: Tuesday, March 03, 2015 4:22 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [nova] what's the merge plan for current proposed microversions? Bump. I'd really appreciate some answers to the question Sean asked. I still have the 2.4 in my review (the very one Sean mentioned) but it seems that it might not be the case. Best regards, Alex Levine On 3/2/15 2:30 PM, Sean Dague wrote: This change for the additional attributes for ec2 looks like it's basically ready to go, except it has the wrong microversion on it (as they anticipated the other changes landing ahead of them) - https://review.openstack.org/#/c/155853 What's the plan for merging the outstanding microversions? I believe we're all conceptually approved on all them, and it's an important part of actually moving forward on the new API. It seems like we're in a bit of a holding pattern on all of them right now, and I'd like to make sure we start merging them this week so that they have breathing space before the freeze. So the plan for assignment of microversion api numbers is the same as what we currently do for db migration changes - take the next one knowing that you may need to rebase if someone else merges before you. Other suggestions welcome but will have to follow the requirement that they always merge in version order. -Sean __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Temporary freeze on API changes
Hi, The Nova API subgroup is planning on releasing V2.1 on Monday (changing its status from experimental to CURRENT) and merging the first patch which uses microversions on top of 2.1 on the Wednesday. In the meantime we'd appreciate it if reviewers did not +A any patchset which changes the REST API (as it potentially means we'd have to sync changes to 2.1 as well. Any future api changes should only be applied to the v2.1/v3 tree and use the microversions mechanism. When https://review.openstack.org/#/c/140313/ is merged, that will signal that microversions is enabled and we are open for patches to v2.1 microversions. At this point no changes should be accepted to the old v2 api code and anything to the v2.1 code base should be done using the microversions mechanism. There is devref docs on how to use microversions either merged or working its way through gerrit. Regards, Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Nova API meeting
Hi, Just a reminder that the weekly Nova API meeting is being held tomorrow Friday UTC . We encourage cloud operators and those who use the REST API such as SDK developers and others who and are interested in the future of the API to participate. In other timezones the meeting is at: EST 20:00 (Thu) Japan 09:00 (Fri) China 08:00 (Fri) ACDT 10:30 (Fri) The proposed agenda and meeting details are here: https://wiki.openstack.org/wiki/Meetings/NovaAPI Please feel free to add items to the agenda. We will also be discussing when to release v2.1 and microversions. I'm going to propose that v2.1 becomes non experimental on Monday and microversions is enabled with the first api change to use it on Wednesday. Please yell here or at the meeting if you think thats a bad idea. Note that the old v2 code is going to remain the default on /v2 so you still need to opt-in to v2.1 and microversioned changes will only affect you if you send the appropriate header Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Outcome of the nova FFE meeting for Kilo
On Wed, Feb 18, 2015 at 6:18 AM, Matt Riedemann mrie...@linux.vnet.ibm.com wrote: On 2/16/2015 9:57 PM, Jay Pipes wrote: Hi Mikal, sorry for top-posting. What was the final decision regarding the instance tagging work? Thanks, -jay On 02/16/2015 09:44 PM, Michael Still wrote: Hi, we had a meeting this morning to try and work through all the FFE requests for Nova. The meeting was pretty long -- two hours or so -- and we did in in the nova IRC channel in an attempt to be as open as possible. The agenda for the meeting was the list of FFE requests at https://etherpad.openstack.org/p/kilo-nova-ffe-requests I recognise that this process is difficult for all, and that it is frustrating when your FFE request is denied. However, we have tried very hard to balance distractions from completing priority tasks and getting as many features into Kilo as possible. I ask for your patience as we work to finalize the Kilo release. That said, here's where we ended up: Approved: vmware: ephemeral disk support API: Keypair support for X509 public key certificates We were also presented with a fair few changes which are relatively trivial (single patch, not very long) and isolated to a small part of the code base. For those, we've selected the ones with the greatest benefit. These ones are approved so long as we can get the code merged before midnight on 20 February 2015 (UTC). The deadline has been introduced because we really are trying to focus on priority work and bug fixes for the remainder of the release, so I want to time box the amount of distraction these patches cause. Those approved in this way are: ironic: Pass the capabilities to ironic node instance_info libvirt: Nova vif driver plugin for opencontrail libvirt: Quiescing filesystems with QEMU guest agent during image snapshotting libvirt: Support vhost user in libvirt vif driver libvirt: Support KVM/libvirt on System z (S/390) as a hypervisor platform It should be noted that there was one request which we decided didn't need a FFE as it isn't feature work. That may proceed: hyperv: unit tests refactoring Finally, there were a couple of changes we were uncomfortable merging this late in the release as we think they need time to bed down before a release we consider stable for a long time. We'd like to see these merge very early in Liberty: libvirt: use libvirt storage pools libvirt: Generic Framework for Securing VNC and SPICE Proxy-To-Compute-Node Connections Thanks again to everyone with their patience with our process, and helping to make Kilo an excellent Nova release. Michael __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev There are notes in the etherpad, https://etherpad.openstack.org/p/kilo-nova-ffe-requests but I think we wanted to get cyeoh and Ken'ichi's thoughts on the v2 and/or v2.1 question about the change, i.e. should it be v2.1 only with microversions or if that is going to block it, is it fair to keep out the v2 change that's already in the patch? So if it can be fully merged by end of week I'm ok with it going into v2 and v2.1. Otherwise I think it needs to wait for microversions. I'd like to see v2.1 enabled next Monday (I don't want it go in just before a weekend). And the first microversion change (which is ready to go) a couple of days after). And we want a bit of an API freeze while that is happening. Chris -- Thanks, Matt Riedemann __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Nova API meeting
Hi, Just a reminder that the weekly Nova API meeting is being held tomorrow Friday UTC . We encourage cloud operators and those who use the REST API such as SDK developers and others who and are interested in the future of the API to participate. In other timezones the meeting is at: EST 20:00 (Thu) Japan 09:00 (Fri) China 08:00 (Fri) ACDT 10:30 (Fri) The proposed agenda and meeting details are here: https://wiki.openstack.org/wiki/Meetings/NovaAPI Please feel free to add items to the agenda. Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Feature Freeze Exception request for x509 keypairs
I'm happy to sponsor this. I've reviewed all the patches as well and as Claudiu mentions we have this lined up as being the first api change to use microversions Regards, Chris On Thu, Feb 12, 2015 at 10:50 PM, Claudiu Belu cb...@cloudbasesolutions.com wrote: Hello. I would like to ask for a FFE for the x509 keypairs blueprint: https://blueprints.launchpad.net/nova/+spec/keypair-x509-certificates This blueprint is split up into 3 commits: [1] Database migration: previously merged, but had to be reverted because of a small issue. Everything is fixed, original reverter Johannes Erdfelt gave his +1, currently the commit has a +2. https://review.openstack.org/#/c/150800/ [2] Nova-API change: It uses the microversioning API and it has been decided to be the first microversioning commit, since it is closest to merge. Christopher Yeoh reviewed helped with this commit. https://review.openstack.org/#/c/140313/ [3] X509 keypair implementation: Simple commit, it had a +2 on a previous commit. https://review.openstack.org/#/c/136869/ I also want to point out that this blueprint targets all the drivers, not just Hyper-V. This blueprint targets all the users that desire to deploy instances with Windows guests and desire password-less authentication, the same way users can ssh into Linux-type guests. Best regards, Claudiu Belu __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cross-Repo Dependencies in Zuul
Wow, this is great! Thank you!
Chris
On Wed, Feb 11, 2015 at 8:56 AM, James E. Blair cor...@inaugust.com wrote:
Hi,
We have added support for cross-repo dependencies (CRD) in Zuul. The
important bits:
* To use them, include Depends-On: gerrit-change-id in the footer of
your commit message. Use the full Change-ID ('I' + 40 characters).
* These are one-way dependencies only -- do not create a cycle.
* This is what all the grey dots and lines are in the check pipeline.
Cross-Repo Dependencies Explained
=
There are two behaviors that might go by the name cross-repo
dependencies. We call them one-way and multi-way.
Multi-way CRD allow for bidirectional links between changes. For
instance, A depends on B and B depends on A. The theory there is that
both would be tested together and merged as a unit. This is _not_ what
we have implemented in Zuul. Discussions over the past two years have
revealed that this type of behavior could cause problems for continuous
deploments as it means that two components must be upgraded
simultaneously. Not supporting this behavior is a choice we have made.
One-way CRD behaves like a directed acyclic graph (DAG), like git
itself, to indicate a one-way dependency relationship between changes in
different git repos. Change A may depend on B, but B may not depend on
A. This is what we have implemented in Zuul.
Gate Pipeline
=
When Zuul sees CRD changes, it serializes them in the usual manner when
enqueuing them into a pipeline. This means that if change A depends on
B, then when they are added to the gate pipeline, B will appear first
and A will follow. If tests for B fail, both B and A will be removed
from the pipeline, and it will not be possible for A to merge until B
does.
Note that if changes with CRD do not share a change queue (such as the
integrated gate then Zuul is unable to enqueue them together, and the
first will be required to merge before the second is enqueued.
Check Pipeline
==
When changes are enqueued into the check pipeline, all of the related
dependencies (both normal git-dependencies that come from parent commits
as well as CRD changes) appear in a dependency graph, as in gate. This
means that even in the check pipeline, your change will be tested with
its dependency. So changes that were previously unable to be fully
tested until a related change landed in a different repo may now be
tested together from the start.
All of the changes are still independent (so you will note that the
whole pipeline does not share a graph as in gate), but for each change
tested, all of its dependencies are visually connected to it, and they
are used to construct the git references that Zuul uses when testing.
When looking at this graph on the status page, you will note that the
dependencies show up as grey dots, while the actual change tested shows
up as red or green. This is to indicate that the grey changes are only
there to establish dependencies. Even if one of the dependencies is
also being tested, it will show up as a grey dot when used as a
dependency, but separately and additionally will appear as its own red
or green dot for its test.
(If you don't see grey dots on the status page, reload the page to get
the latest version.)
Multiple Changes
A Gerrit change ID may refer to multiple changes (on multiple branches
of the same project, or even multiple projects). In these cases, Zuul
will treat all of the changes with that change ID as dependencies. So
if you say that a tempest change Depends-On a change ID that has changes
in nova master and nova stable/juno, then when testing the tempest
change, both nova changes will be applied, and when deciding whether the
tempest change can merge, both changes must merge ahead of it.
A change may depend on more than one Gerrit change ID as well. So it is
possible for a change in tempest to depend on a change in devstack and a
change in nova. Simply add more Depends-On: lines to the footer.
Cycles
==
If a cycle is created by use of CRD, Zuul will abort its work very
early. There will be no message in Gerrit and no changes that are part
of the cycle will be enqueued into any pipeline. This is to protect
Zuul from infinite loops. I hope that we can improve this to at least
leave a message in Gerrit in the future. But in the meantime, please be
cognizant of this and do not create dependency cycles with Depends-On
lines.
Examples
The following two infra changes have been tested together because of the
Depends-On: line in the commit message of the first:
https://review.openstack.org/#/c/152508/
https://review.openstack.org/#/c/152504/
In fact, you can see earlier test results failing until it was rechecked
after CRD went into production (around 2015-02-10 15:20 UTC).
This devstack change depended on a grenade change:
Re: [openstack-dev] [nova] will the real v2.1/v3 API status please stand up?
Hi, On Sat, Feb 7, 2015 at 8:35 AM, Matt Riedemann mrie...@linux.vnet.ibm.com wrote: I'm not going to hide it, I don't know what's going on with the v2.1 API status, i.e. what is the criteria to that thing dropping it's 'experimental' label? So I caught up with Matt on IRC, repeating some references and discussion here for everyone else I wasn't at the mid-cycle meetup for Kilo but even for Juno I'll admit I was a bit lost. It's not my fault, I'm more good looks than brains. :) When I look at approved specs for Kilo, three pop out: 1. https://blueprints.launchpad.net/nova/+spec/v2-on-v3-api 2. https://blueprints.launchpad.net/nova/+spec/api-microversions 3. https://blueprints.launchpad.net/nova/+spec/v3-api-policy So we need the first to blueprints for v2.1 microversions. We don't need v3-api-policy merged to release v2.1 microversions though I believe it is a separate important bit of work to reduce tech debt and make life easier for operators. The only one of those that has a dependency in launchpad is the last one and it's dependency is on: https://blueprints.launchpad.net/nova/+spec/nova-v3-api Which looks like it was replaced by the v2-on-v3-api blueprint. If I look at the open changes for each, there are a lot: 1. https://review.openstack.org/#/q/status:open+project: openstack/nova+branch:master+topic:bp/v2-on-v3-api,n,z 2. https://review.openstack.org/#/q/status:open+project: openstack/nova+branch:master+topic:bp/api-microversions,n,z 3. https://review.openstack.org/#/q/status:open+project: openstack/nova+branch:master+topic:bp/v3-api-policy,n,z Do those all need to merge before the v2.1 API is no longer experimental? We have an etherpad here which tracks our release criteria for v2.1 and microversions: https://etherpad.openstack.org/p/v2_1_ReleaseCriteria As mentioned above it doesn't include api-policy To make life easier for us I'd also like to request that if you review a changeset that modifies the v2 api that you ensure it also if required is applied to v2.1(v3 code). If it doesn't apply to v3 then ensure a v2-only tag is in the commit message. That will help us verify v2 does not diverge from v2.1 just before release. After that I think v2 code will be essentially frozen except for bug fixes and any api changes will only be made through microversions. Regards, Chris Is the, for lack of a better term, 'completion criteria', being tracked in an etherpad or wiki page somewhere? I see stuff in the priorities etherpad https://etherpad.openstack.org/p/kilo-nova-priorities-tracking but it's not clear to me at a high level what makes v2.1 no longer experimental. Can someone provide that in less than 500 words? -- Thanks, Matt Riedemann __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [python-novaclient][nova] future of --os-compute-api-version option and whole api versioning
On Thu, Feb 5, 2015 at 11:37 AM, Andrey Kurilin akuri...@mirantis.com wrote: First results is https://review.openstack.org/#/c/152569/ - if os-compute-api-version is not supplied don't send any header at all - it is probably worth doing a bit version parsing to see if it makes sense eg of format: r^([1-9]\d*)\.([1-9]\d*|0)$ or latest implemented - handle HTTPNotAcceptable if the user asked for a version which is not supported (can also get a badrequest if its badly formatted and got through the novaclient filter) Based on https://review.openstack.org/#/c/150641/ , HTTPNotFound (404) will be returned by API, so the current implementation of novaclient is not required any changes. So a microversion (say v2.12) is a concept which covers the API as a whole, not just an extension. And as a result there is a concept of a minimum version supported (currently 2.1) and a maximum). If a client requests a version which it outside of min-max they will have a HTTPNotAcceptable returned. If a client a requests a version which is valid, but the actual resource point they attempt to access does not support an implementation (say the version requested did not support anything yet), they will get a 404 (eg it pretends not to be there). - show the version header information returned Imo, API should return exception with proper message which already include this information. Yep, if you request a version which doesn't fit within the global supported (min-max) you will get a 406 which specifies min/max api versions for that server. For any other request you get returned an X-OpenStack-Compute-API-Version header which specifies the version used - which you may nto have exactly known on the request if you didn't specify a version (eg not specified or 'latest') Regards, Chris On Mon, Feb 2, 2015 at 2:02 PM, Andrey Kurilin akuri...@mirantis.com wrote: Thanks for the summary, I'll try to send first patch(maybe WIP) in few days. On Mon, Feb 2, 2015 at 1:43 PM, Christopher Yeoh cbky...@gmail.com wrote: On Sat, Jan 31, 2015 at 4:09 AM, Andrey Kurilin akuri...@mirantis.com wrote: Thanks for the answer. Can I help with implementation of novaclient part? Sure! Do you think its something you can get proposed into Gerrit by the end of the week or very soon after? Its the sort of timeframe we're looking for to get microversions enabled asap I guess just let me know if it turns out you don't have the time. So I think a short summary of what is needed is: - if os-compute-api-version is not supplied don't send any header at all - it is probably worth doing a bit version parsing to see if it makes sense eg of format: r^([1-9]\d*)\.([1-9]\d*|0)$ or latest - handle HTTPNotAcceptable if the user asked for a version which is not supported (can also get a badrequest if its badly formatted and got through the novaclient filter) - show the version header information returned Regards, Chris On Wed, Jan 28, 2015 at 11:50 AM, Christopher Yeoh cbky...@gmail.com wrote: On Fri, 23 Jan 2015 15:51:54 +0200 Andrey Kurilin akuri...@mirantis.com wrote: Hi everyone! After removing nova V3 API from novaclient[1], implementation of v1.1 client is used for v1.1, v2 and v3 [2]. Since we moving to micro versions, I wonder, do we need such mechanism of choosing api version(os-compute-api-version) or we can simply remove it, like in proposed change - [3]? If we remove it, how micro version should be selected? So since v3 was never officially released I think we can re-use os-compute-api-version for microversions which will map to the X-OpenStack-Compute-API-Version header. See here for details on what the header will look like. We need to also modify novaclient to handle errors when a version requested is not supported by the server. If the user does not specify a version number then we should not send anything at all. The server will run the default behaviour which for quite a while will just be v2.1 (functionally equivalent to v.2) http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/api-microversions.html [1] - https://review.openstack.org/#/c/138694 [2] - https://github.com/openstack/python-novaclient/blob/master/novaclient/client.py#L763-L769 [3] - https://review.openstack.org/#/c/149006 -- Best regards, Andrey Kurilin. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best regards, Andrey Kurilin. -- Best regards, Andrey Kurilin. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http
Re: [openstack-dev] [api][nova] Openstack HTTP error codes
On Tue, Feb 3, 2015 at 9:05 AM, Jay Pipes jaypi...@gmail.com wrote:
On 01/29/2015 12:41 PM, Sean Dague wrote:
Correct. This actually came up at the Nova mid cycle in a side
conversation with Ironic and Neutron folks.
HTTP error codes are not sufficiently granular to describe what happens
when a REST service goes wrong, especially if it goes wrong in a way
that would let the client do something other than blindly try the same
request, or fail.
Having a standard json error payload would be really nice.
{
fault: ComputeFeatureUnsupportedOnInstanceType,
messsage: This compute feature is not supported on this kind of
instance type. If you need this feature please use a different instance
type. See your cloud provider for options.
}
That would let us surface more specific errors.
snip
Standardization here from the API WG would be really great.
What about having a separate HTTP header that indicates the OpenStack
Error Code, along with a generated URI for finding more information about
the error?
Something like:
X-OpenStack-Error-Code: 1234
X-OpenStack-Error-Help-URI: http://errors.openstack.org/1234
That way is completely backwards compatible (since we wouldn't be changing
response payloads) and we could handle i18n entirely via the HTTP help
service running on errors.openstack.org.
So I'm +1 to adding the X-OpenStack-Error-Code header assuming the error
code is unique
across OpenStack APIs and it has a fixed meaning (we never change it,
create a new one if
a project has a need for an error code which is close to the original one
but a bit different)
The X-OpenStack-Error-Help-URI header I'm not sure about. We can't
guarantee that apps will have
access to errors.openstack.org - is there an assumption here that we'd
build/ship an error translation service?
Regards,
Chris
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [python-novaclient][nova] future of --os-compute-api-version option and whole api versioning
On Sat, Jan 31, 2015 at 4:09 AM, Andrey Kurilin akuri...@mirantis.com wrote: Thanks for the answer. Can I help with implementation of novaclient part? Sure! Do you think its something you can get proposed into Gerrit by the end of the week or very soon after? Its the sort of timeframe we're looking for to get microversions enabled asap I guess just let me know if it turns out you don't have the time. So I think a short summary of what is needed is: - if os-compute-api-version is not supplied don't send any header at all - it is probably worth doing a bit version parsing to see if it makes sense eg of format: r^([1-9]\d*)\.([1-9]\d*|0)$ or latest - handle HTTPNotAcceptable if the user asked for a version which is not supported (can also get a badrequest if its badly formatted and got through the novaclient filter) - show the version header information returned Regards, Chris On Wed, Jan 28, 2015 at 11:50 AM, Christopher Yeoh cbky...@gmail.com wrote: On Fri, 23 Jan 2015 15:51:54 +0200 Andrey Kurilin akuri...@mirantis.com wrote: Hi everyone! After removing nova V3 API from novaclient[1], implementation of v1.1 client is used for v1.1, v2 and v3 [2]. Since we moving to micro versions, I wonder, do we need such mechanism of choosing api version(os-compute-api-version) or we can simply remove it, like in proposed change - [3]? If we remove it, how micro version should be selected? So since v3 was never officially released I think we can re-use os-compute-api-version for microversions which will map to the X-OpenStack-Compute-API-Version header. See here for details on what the header will look like. We need to also modify novaclient to handle errors when a version requested is not supported by the server. If the user does not specify a version number then we should not send anything at all. The server will run the default behaviour which for quite a while will just be v2.1 (functionally equivalent to v.2) http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/api-microversions.html [1] - https://review.openstack.org/#/c/138694 [2] - https://github.com/openstack/python-novaclient/blob/master/novaclient/client.py#L763-L769 [3] - https://review.openstack.org/#/c/149006 -- Best regards, Andrey Kurilin. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Product] [all][log] Openstack HTTP error codes
On Sun, Feb 1, 2015 at 2:57 AM, Sean Dague s...@dague.net wrote:
On 01/31/2015 05:24 AM, Duncan Thomas wrote:
Hi
This discussion came up at the cinder mid-cycle last week too,
specifically in the context of 'Can we change the details text in an
existing error, or is that an unacceptable API change'.
I have to second security / operational concerns about exposing too much
granularity of failure in these error codes.
For cases where there is something wrong with the request (item out of
range, invalid names, feature not supported, etc) I totally agree that
we should have good, clear, parsable response, and standardisation would
be good. Having some fixed part of the response (whether a numeric code
or, as I tend to prefer, a CamelCaseDescription so that I don't have to
go look it up) and a human readable description section that is subject
to change seems sensible.
What I would rather not see is leakage of information when something
internal to the cloud goes wrong, that the tenant can do nothing
against. We certainly shouldn't be leaking internal implementation
details like vendor details - that is what request IDs and logs are for.
The whole point of the cloud, to me, is that separation between the
things a tenant controls (what they want done) and what the cloud
provider controls (the details of how the work is done).
For example, if a create volume request fails because cinder-scheduler
has crashed, all the tenant should get back is 'Things are broken, try
again later or pass request id 1234-5678-abcd-def0 to the cloud admin'.
They should need to or even be allowed to care about the details of the
failure, it is not their domain.
Sure, the value really is in determining things that are under the
client's control to do differently. A concrete one is a multi hypervisor
cloud with 2 hypervisors (say kvm and docker). The volume attach
operation to a docker instance (which presumably is a separate set of
instance types) can't work. The user should be told that that can't work
with this instance_type if they try it.
That's actually user correctable information. And doesn't require a
ticket to move forward.
I also think we could have a detail level knob, because I expect the
level of information exposure might be considered different in public
cloud use case vs. a private cloud at an org level or a private cloud at
a dept level.
That could turn into a major compatibility issue if what we returned could
(and
probably would between public/private) change between clouds? If we want to
encourage
people to parse this sort of thing I think we need to settle on whether we
send the
information back or not for everyone.
-Sean
On 30 January 2015 at 02:34, Rochelle Grober rochelle.gro...@huawei.com
mailto:rochelle.gro...@huawei.com wrote:
Hi folks!
Changed the tags a bit because this is a discussion for all projects
and dovetails with logging rationalization/standards/
At the Paris summit, we had a number of session on logging that kept
circling back to Error Codes. But, these codes would not be http
codes, rather, as others have pointed out, codes related to the
calling entities and referring entities and the actions that
happened or didn’t. Format suggestions were gathered from the
Operators and from some senior developers. The Logging Working
Group is planning to put forth a spec for discussion on formats and
standards before the Ops mid-cycle meetup.
Working from a Glance proposal on error codes:
https://review.openstack.org/#/c/127482/ and discussions with
operators and devs, we have a strawman to propose. We also have a
number of requirements from Ops and some Devs.
Here is the basic idea:
Code for logs would have four segments:
Project Vendor/Component Error
Catalog number Criticality
Def [A-Z] [A-Z] [A-Z] -
[{0-9}|{A-Z}][A-Z] - [-]- [0-9]
Ex. CIN- NA-
0001-
2
Cinder NetApp
driver error no
Criticality
Ex. GLA- 0A-
0051
3
Glance Api
error no
Criticality
Three letters for project, Either a two letter vendor code or a
number and letter for 0+letter for internal component of project
(like API=0A, Controller =0C, etc), four digit error number which
could be subsetted for even finer granularity, and a criticality
number.
This is for logging
Re: [openstack-dev] [nova] novaclient support for V2.1 micro versions
On Fri, 23 Jan 2015 16:53:55 + Day, Phil philip@hp.com wrote: Hi Folks, Is there any support yet in novaclient for requesting a specific microversion ? (looking at the final leg of extending clean-shutdown to the API, and wondering how to test this in devstack via the novaclient) No, sorry, something should up within a week. Regards, Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [python-novaclient][nova] future of --os-compute-api-version option and whole api versioning
On Fri, 23 Jan 2015 15:51:54 +0200 Andrey Kurilin akuri...@mirantis.com wrote: Hi everyone! After removing nova V3 API from novaclient[1], implementation of v1.1 client is used for v1.1, v2 and v3 [2]. Since we moving to micro versions, I wonder, do we need such mechanism of choosing api version(os-compute-api-version) or we can simply remove it, like in proposed change - [3]? If we remove it, how micro version should be selected? So since v3 was never officially released I think we can re-use os-compute-api-version for microversions which will map to the X-OpenStack-Compute-API-Version header. See here for details on what the header will look like. We need to also modify novaclient to handle errors when a version requested is not supported by the server. If the user does not specify a version number then we should not send anything at all. The server will run the default behaviour which for quite a while will just be v2.1 (functionally equivalent to v.2) http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/api-microversions.html [1] - https://review.openstack.org/#/c/138694 [2] - https://github.com/openstack/python-novaclient/blob/master/novaclient/client.py#L763-L769 [3] - https://review.openstack.org/#/c/149006 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Questions on pep8 F811 hacking check for microversion
On Tue, 06 Jan 2015 07:31:19 -0500 Jay Pipes jaypi...@gmail.com wrote: On 01/06/2015 06:25 AM, Chen CH Ji wrote: Based on nova-specs api-microversions.rst we support following function definition format, but it violate the hacking rule pep8 F811 because duplicate function definition we should use #noqa for them , but considering microversion may live for long time , keep adding #noqa may be a little bit ugly, can anyone suggest a good solution for it ? thanks @api_version(min_version='2.1') def _version_specific_func(self, req, arg1): pass @api_version(min_version='2.5') def _version_specific_func(self, req, arg1): pass Hey Kevin, This was actually one of my reservations about the proposed microversioning implementation -- i.e. having functions that are named exactly the same, only decorated with the microversioning notation. It kinda reminds me of the hell of debugging C++ code that uses STL: how does one easily know which method one is in when inside a debugger? That said, the only other technique we could try to use would be to not use a decorator and instead have a top-level dispatch function that would inspect the API microversion (only when the API version makes a difference to the output or input of that function) and then dispatch the call to a helper method that had the version in its name. So, for instance, let's say you are calling the controller's GET /$tenant/os-hosts method, which happens to get routed to the nova.api.openstack.compute.contrib.hosts.HostController.index() method. If you wanted to modify the result of that method and the API microversion is at 2.5, you might do something like: def index(self, req): req_api_ver = utils.get_max_requested_api_version(req) if req_api_ver == (2, 5): return self.index_2_5(req) return self.index_2_1(req) def index_2_5(self, req): results = self.index_2_1(req) # Replaces 'host' with 'host_name' for result in results: result['host_name'] = result['host'] del result['host'] return results def index_2_1(self, req): # Would be a rename of the existing index() method on # the controller So having to manually add switching code everything we have an API patch I think is not only longer and more complicated but more error prone when updating. If we change something at the core in the future it means changing all the microversioned code rather than just the switching architecture at the core of wsgi. Another option would be to use something like JSON-patch to determine the difference between two output schemas and automatically translate one to another... but that would be a huge effort. That's the only other way I can think of besides disabling F811, which I really would not recommend, since it's a valuable safeguard against duplicate function names (especially duplicated test methods). So I don't think we need to disable F811 in general - why not just disable it for any method with the api_version decorator? On those ones we can do checks on what is passed to api_version which will help verify that there hasn't been a typo to an api_version decorator. Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] [nova] [glance] Consistency in client side sorting
On Mon, 05 Jan 2015 11:10:41 -0500 Jay Pipes jaypi...@gmail.com wrote: Thoughts on getting consistency across all 3 projects (and possibly others)? Yeah, I personally like the second option as well, but agree that consistency is the key (pun intended) here. I would say let's make a decision on the standard to go with (possibly via the API or SDK working groups?) and then move forward with support for that option in all three clients (and continue to support the old behaviour for 2 release cycles, with deprecation markings as appropriate). +1 to making this available in a consistent way. We need need to support the old client behaviour for at least a couple of cycles (maybe a bit longer) and the burden of doing so is pretty low. I don't think however that we can drop the REST API behaviour that quickly. 2 cycles for API deprecation in the past has been considered an insufficent length of time because of app breakage Regards, Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Nominating Melanie Witt for python-novaclient-core
On Wed, Jan 28, 2015 at 9:11 AM, Michael Still mi...@stillhq.com wrote: Greetings, I would like to nominate Melanie Witt for the python-novaclient-core team. (What is python-novaclient-core? Its a new group which will contain all of nova-core as well as anyone else we think should have core reviewer powers on just the python-novaclient code). Melanie has been involved with nova for a long time now. She does solid reviews in python-novaclient, and at least two current nova-cores have suggested her as ready for core review powers on that repository. Please respond with +1s or any concerns. +1 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Complexity check and v2 API
Hi,
Given the timing (no spec approved) it sounds like a v2.1 plus
microversions (just merging) with no v2 changes at all.
The v2.1 framework is more flexible and you should need no changes to
servers.py at all as there are hooks for adding extra parameters in
separate plugins. There are examples of this in the v3 directory which is
really v2.1 now.
Chris
On Thu, 18 Dec 2014 at 3:49 am, Pasquale Porreca
pasquale.porr...@dektech.com.au wrote:
Thank you for the answer.
my API proposal won't be merged in kilo release since the deadline for
approval is tomorrow, so I may propose the fix to lower the complexity
in another way, what do you think about a bug fix?
On 12/17/14 18:05, Matthew Gilliard wrote:
Hello Pasquale
The problem is that you are trying to add a new if/else branch into
a method which is already ~250 lines long, and has the highest
complexity of any function in the nova codebase. I assume that you
didn't contribute much to that complexity, but we've recently added a
limit to stop it getting any worse. So, regarding your 4 suggestions:
1/ As I understand it, v2.1 should be the same as v2 at the
moment, so they need to be kept the same
2/ You can't ignore it - it will fail CI
3/ No thank you. This limit should only ever be lowered :-)
4/ This is 'the right way'. Your suggestion for the refactor does
sound good.
I suggest a single patch that refactors and lowers the limit in
tox.ini. Once you've done that then you can add the new parameter in
a following patch. Please feel free to add me to any patches you
create.
Matthew
On Wed, Dec 17, 2014 at 4:18 PM, Pasquale Porreca
pasquale.porr...@dektech.com.au wrote:
Hello
I am working on an API extension that adds a parameter on create server
call; to implement the v2 API I added few lines of code to
nova/api/openstack/compute/servers.py
In particular just adding something like
new_param = None
if self.ext_mgr.is_loaded('os-new-param'):
new_param = server_dict.get('new_param')
leads to a pep8 fail with message 'Controller.create' is too complex
(47)
(Note that in tox.ini the max complexity is fixed to 47 and there is a
note
specifying 46 is the max complexity present at the moment).
It is quite easy to make this test pass creating a new method just to
execute these lines of code, anyway all other extensions are handled in
that
way and one of most important stylish rule states to be consistent with
surrounding code, so I don't think a separate function is the way to go
(unless it implies a change in how all other extensions are handled
too).
My thoughts on this situation:
1) New extensions should not consider v2 but only v2.1, so that file
should
not be touched
2) Ignore this error and go on: if and when the extension will be
merged the
complexity in tox.ini will be changed too
3) The complexity in tox.ini should be raised to allow new v2 extensions
4) The code of that module should be refactored to lower the complexity
(i.e. move the load of each extension in a separate function)
I would like to know if any of my point is close to the correct
solution.
--
Pasquale Porreca
DEK Technologies
Via dei Castelli Romani, 22
00040 Pomezia (Roma)
Mobile +39 3394823805
Skype paskporr
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Pasquale Porreca
DEK Technologies
Via dei Castelli Romani, 22
00040 Pomezia (Roma)
Mobile +39 3394823805
Skype paskporr
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] UniqueConstraint for name and tenant_id in security group
So I think this is something we really should get agreement on across the open stack API first before flipping back and forth on a case by case basis. Personally I think we should be using uuids for uniqueness and leave any extra restrictions to a ui layer if really required. If we try to have name uniqueness then test should be considered the same as test as test and it introduces all sorts of slightly different combos that look the same except under very close comparison. Add unicode for extra fun. Chris On Tue, 16 Dec 2014 at 7:24 am, Maru Newby ma...@redhat.com wrote: On Dec 15, 2014, at 9:13 AM, Assaf Muller amul...@redhat.com wrote: - Original Message - -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 I was (rightfully) asked to share my comments on the matter that I left in gerrit here. See below. On 12/12/14 22:40, Sean Dague wrote: On 12/12/2014 01:05 PM, Maru Newby wrote: On Dec 11, 2014, at 2:27 PM, Sean Dague s...@dague.net wrote: On 12/11/2014 04:16 PM, Jay Pipes wrote: On 12/11/2014 04:07 PM, Vishvananda Ishaya wrote: On Dec 11, 2014, at 1:04 PM, Jay Pipes jaypi...@gmail.com wrote: On 12/11/2014 04:01 PM, Vishvananda Ishaya wrote: On Dec 11, 2014, at 8:00 AM, Henry Gessau ges...@cisco.com wrote: On Thu, Dec 11, 2014, Mark McClain m...@mcclain.xyz wrote: On Dec 11, 2014, at 8:43 AM, Jay Pipes jaypi...@gmail.com mailto:jaypi...@gmail.com wrote: I'm generally in favor of making name attributes opaque, utf-8 strings that are entirely user-defined and have no constraints on them. I consider the name to be just a tag that the user places on some resource. It is the resource's ID that is unique. I do realize that Nova takes a different approach to *some* resources, including the security group name. End of the day, it's probably just a personal preference whether names should be unique to a tenant/user or not. Maru had asked me my opinion on whether names should be unique and I answered my personal opinion that no, they should not be, and if Neutron needed to ensure that there was one and only one default security group for a tenant, that a way to accomplish such a thing in a race-free way, without use of SELECT FOR UPDATE, was to use the approach I put into the pastebin on the review above. I agree with Jay. We should not care about how a user names the resource. There other ways to prevent this race and Jay’s suggestion is a good one. However we should open a bug against Horizon because the user experience there is terrible with duplicate security group names. The reason security group names are unique is that the ec2 api supports source rule specifications by tenant_id (user_id in amazon) and name, so not enforcing uniqueness means that invocation in the ec2 api will either fail or be non-deterministic in some way. So we should couple our API evolution to EC2 API then? -jay No I was just pointing out the historical reason for uniqueness, and hopefully encouraging someone to find the best behavior for the ec2 api if we are going to keep the incompatibility there. Also I personally feel the ux is better with unique names, but it is only a slight preference. Sorry for snapping, you made a fair point. Yeh, honestly, I agree with Vish. I do feel that the UX of that constraint is useful. Otherwise you get into having to show people UUIDs in a lot more places. While those are good for consistency, they are kind of terrible to show to people. While there is a good case for the UX of unique names - it also makes orchestration via tools like puppet a heck of a lot simpler - the fact is that most OpenStack resources do not require unique names. That being the case, why would we want security groups to deviate from this convention? Maybe the other ones are the broken ones? Honestly, any sanely usable system makes names unique inside a container. Like files in a directory. Correct. Or take git: it does not use hashes to identify objects, right? In this case the tenant is the container, which makes sense. It is one of many places that OpenStack is not consistent. But I'd rather make things consistent and more usable than consistent and less. Are we only proposing to make security group name unique? I assume that, since that's what we currently have in review. The change would make API *more* inconsistent, not less, since other objects still use uuid for identification. You may say that we should move *all* neutron objects to the new identification system by name. But what's the real benefit? If there are problems in UX (client, horizon, ...), we should fix the view and not data models used. If we decide we want users to avoid using objects with the same names, fine, let's add warnings in UI (probably with an option to disable it so that we don't push
Re: [openstack-dev] [qa] How to delete a VM which is in ERROR state?
If force delete doesn't work please do submit the bug report along with as much of the relevant nova logs as you can. Even better if it's easily repeatable with devstack. Chris On Sat, 13 Dec 2014 at 8:43 am, pcrews glee...@gmail.com wrote: On 12/09/2014 03:54 PM, Ken'ichi Ohmichi wrote: Hi, This case is always tested by Tempest on the gate. https://github.com/openstack/tempest/blob/master/tempest/ api/compute/servers/test_delete_server.py#L152 So I guess this problem wouldn't happen on the latest version at least. Thanks Ken'ichi Ohmichi --- 2014-12-10 6:32 GMT+09:00 Joe Gordon joe.gord...@gmail.com: On Sat, Dec 6, 2014 at 5:08 PM, Danny Choi (dannchoi) dannc...@cisco.com wrote: Hi, I have a VM which is in ERROR state. +--+ --+++--- --++ | ID | Name | Status | Task State | Power State | Networks | +--+ --+++--- --++ | 1cb5bf96-619c-4174-baae-dd0d8c3d40c5 | cirros--1cb5bf96-619c-4174-baae-dd0d8c3d40c5 | ERROR | - | NOSTATE || I tried in both CLI “nova delete” and Horizon “terminate instance”. Both accepted the delete command without any error. However, the VM never got deleted. Is there a way to remove the VM? What version of nova are you using? This is definitely a serious bug, you should be able to delete an instance in error state. Can you file a bug that includes steps on how to reproduce the bug along with all relevant logs. bugs.launchpad.net/nova Thanks, Danny ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Hi, I've encountered this in my own testing and have found that it appears to be tied to libvirt. When I hit this, reset-state as the admin user reports success (and state is set), *but* things aren't really working as advertised and subsequent attempts to do anything with the errant vm's will send them right back into 'FLAIL' / can't delete / endless DELETING mode. restarting libvirt-bin on my machine fixes this - after restart, the deleting vm's are properly wiped without any further user input to nova/horizon and all seems right in the world. using: devstack ubuntu 14.04 libvirtd (libvirt) 1.2.2 triggered via: lots of random create/reboot/resize/delete requests of varying validity and sanity. Am in the process of cleaning up my test code so as not to hurt anyone's brain with the ugly and will file a bug once done, but thought this worth sharing. Thanks, Patrick ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [python-novaclient] Status of novaclient V3
On 3 Dec 2014, at 10:00 pm, Joe Gordon joe.gord...@gmail.com wrote: On Tue, Dec 2, 2014 at 5:21 PM, Andrey Kurilin akuri...@mirantis.com wrote: Hi! While working on fixing wrong import in novaclient v3 shell, I have found that a lot of commands, which are listed in V3 shell(novaclient.v3.shell), are broken, because appropriate managers are missed from V3 client(novaclient.V3.client.Client). Template of error is ERROR (AttributeError): 'Client' object has no attribute 'attr', where attr can be floating_ip_pools, floating_ip, security_groups, dns_entries and etc. I know that novaclient V3 is not finished yet, and I guess it will be not finished. So the main question is: What we should do with implemented code of novaclient V3 ? Should it be ported to novaclient V2.1 or it can be removed? I think it can be removed, as we are not going forward with the V3 API. But I will defer to Christopher Yeoh/Ken’ichi Ohmichi for the details. I think it can all just be removed now. We're going to need to enhance nova client to understand micro versions instead. But for now v2.1 should look just like v2 Chris -- Best regards, Andrey Kurilin. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA][Tempest] Proposing Ghanshyam Mann for Tempest Core
+1 Sent from my iPad On 22 Nov 2014, at 4:56 am, Matthew Treinish mtrein...@kortar.org wrote: Hi Everyone, I'd like to propose we add Ghanshyam Mann (gmann) to the tempest core team. Over the past couple of cycles Ghanshyam has been actively engaged in the Tempest community. Ghanshyam has had one of the highest review counts on Tempest for the past cycle, and he has consistently been providing reviews that have been of consistently high quality that show insight into both the project internals and it's future direction. I feel that Ghanshyam will make an excellent addition to the core team. As per the usual, if the current Tempest core team members would please vote +1 or -1(veto) to the nomination when you get a chance. We'll keep the polls open for 5 days or until everyone has voted. Thanks, Matt Treinish References: https://review.openstack.org/#/q/reviewer:%22Ghanshyam+Mann+%253Cghanshyam.mann%2540nectechnologies.in%253E%22,n,z http://stackalytics.com/?user_id=ghanshyammannmetric=marks ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Counting resources
On Thu, 20 Nov 2014 14:47:16 +0100
Salvatore Orlando sorla...@nicira.com wrote:
Aloha guardians of the API!
I haven recently* reviewed a spec for neutron [1] proposing a
distinct URI for returning resource count on list operations.
This proposal is for selected neutron resources, but I believe the
topic is general enough to require a guideline for the API working
group. Your advice is therefore extremely valuable.
In a nutshell the proposal is to retrieve resource count in the
following way:
GET /prefix/resource_name/count
In my limited experience with RESTful APIs, I've never encountered
one that does counting in this way. This obviously does not mean it's
a bad idea. I think it's not great from a usability perspective to
require two distinct URIs to fetch the first page and then the total
number of elements. I reckon the first response page for a list
operation might include also the total count. For example:
{'resources': [{meh}, {meh}, {meh_again}],
'resource_count': 55
link_to_next_page}
I am however completely open to consider other alternatives.
What is your opinion on this matter?
FWIW there is a nova spec proposed for counting resources as
well (I think it might have been previously approved for Juno).
https://review.openstack.org/#/c/134279/
I haven't compared the two, but I can't think of a reason we'd
need to be any different between projects here.
Regards,
Chris
Regards,
Salvatore
* it's been 10 days now
[1] https://review.openstack.org/#/c/102199/
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Nova API meeting
Hi, Just a reminder that the weekly Nova API meeting is being held tomorrow Friday UTC . We encourage cloud operators and those who use the REST API such as SDK developers and others who and are interested in the future of the API to participate. In other timezones the meeting is at: EST 20:00 (Thu) Japan 09:00 (Fri) China 08:00 (Fri) ACDT 10:30 (Fri) The proposed agenda and meeting details are here: https://wiki.openstack.org/wiki/Meetings/NovaAPI Please feel free to add items to the agenda. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Do we need to add xml support for new API extensions on v2 API ?
On Wed, 19 Nov 2014 13:11:40 +0800 Chen CH Ji jiche...@cn.ibm.com wrote: Hi I saw we are removing v2 XML support proposed several days before For new api extensions, do we need to add it now and remove it later or only support JSON ? Thanks I don't think any api additions to the api should include XML support. Regards, Chris Best Regards! Kevin (Chen) Ji 纪 晨 Engineer, zVM Development, CSTL Notes: Chen CH Ji/China/IBM@IBMCN Internet: jiche...@cn.ibm.com Phone: +86-10-82454158 Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [api] API-WG meeting (note time change this week)
Hi, We have moved to alternating times each week for the API WG meeting so people from other timezones can attend. Since this is an odd week the meeting will be Thursday UTC 1600. Details here: https://wiki.openstack.org/wiki/Meetings/API-WG The google ical feed hasn't been updated yet, but thats not surprising since the wiki page was only updated a few hours ago. Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API-WG meeting (note time change this week)
On Wed, 19 Nov 2014 19:34:44 + Everett Toews everett.to...@rackspace.com wrote: 2. Do you know if there is a way to subscribe to only the API WG meeting from that calendar? I haven't been able to find a way to do that. Fortunately for me most of the openstack meetings end up being between 12am and 8am so it doesn't actually clutter up my calendar view ;-) Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] policy on old / virtually abandoned patches
On Tue, Nov 18, 2014 at 11:44 PM, Jay Pipes jaypi...@gmail.com wrote: On 11/18/2014 07:29 AM, Daniel P. Berrange wrote: On Tue, Nov 18, 2014 at 07:06:59AM -0500, Sean Dague wrote: Nova currently has 197 patches that have seen no activity in the last 4 weeks (project:openstack/nova age:4weeks status:open). Of these * 108 are currently Jenkins -1 (project:openstack/nova age:4weeks status:open label:Verified=-1,jenkins) * 60 are -2 by a core team member (project:openstack/nova age:4weeks status:open label:Code-Review=-2) (note, those 2 groups sometimes overlap) Regardless, the fact that Nova currently has 792 open reviews, and 1/4 of them seem dead, seems like a cleanup thing we could do. I'd like to propose that we implement our own auto abandon mechanism based on reviews that are either held by a -2, or Jenkins -1 after 4 weeks time. I can write a quick script to abandon with a friendly message about why we are doing it, and to restore it if work is continuing. Yep, purging anything that's older than 4 weeks with negative karma seems like a good idea. It'll make it easier for us to identify those patches which are still maintained and target them for review. That said, there's some edge cases - for example I've got some patches up for review that have a -2 on them, becase we're waiting for blueprint approval. IIRC, previously we would post a warning about pending auto- abandon a week before, and thus give the author the chance to add a comment to prevent auto-abandon taking place. It would be neccessary to have this ability to deal with the case where we're just temporarily blocked on other work. Yes, this is indeed an issue. However, couldn't we just say Add a -Workflow label to avoid auto-abandon and then have the script simply ignore patches with -Workflow? +1 for not auto abandoning patches marked WIP (or at least make the timeout much longer for those) Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] v2 or v3 for new api
On Mon, Nov 17, 2014 at 7:30 PM, Pasquale Porreca pasquale.porr...@dektech.com.au wrote: Thank you for the clarification, yes I know about the blueprint/specification, I submitted yet them and the spec is currently under review :) I noticed there are several steps one has always to do to enable and make a v3 api to work and pass the tests. It would be awesome to have a guideline or something similar that explain these steps, but I didn't find anything in wiki or documentation. Yes, sorry documentation has been on our todo list for too long. Could I get you to submit a bug report about the lack of developer documentation for api plugins? It might hurry us up :-) In the meantime, off the top of my head. you'll need to create or modify the following files in a typical plugin: setup.cfg - add an entry in at least the nova.api.v3.extensions section etc/nova/policy.json - an entry for the permissions for you plugin, perhaps one per api method for maximum flexibility. Also will need a discoverable entry (lots of examples in this file) nova/tests/unit/fake_policy.json (similar to policy.json) nova/api/openstack/compute/plugins/v3/your_plugin.py - please make the alias name something os-scheduler-hints rather than OS-SCH-HNTS. No skimping on vowels. Probably the easiest way at this stage without more doco is look for for a plugin in that directory that does the sort of the thing you want to do. nova/tests/unit/nova/api/openstack/compute/contrib/test_your_plugin.py - we have been combining the v2 and v2.1(v3) unittests to share as much as possible, so please do the same here for new tests as the v3 directory will be eventually removed. There's quite a few examples now in that directory of sharing unittests between v2.1 and v2 but with a new extension the customisation between the two should be pretty minimal (just a bit of inheritance to call the right controller) nova/tests/unit/integrated/v3/test_your_plugin.py nova/tests/unit/integrated/test_api_samples.py Sorry the api samples tests are not unified yet. So you'll need to create two. All of the v2 api sample tests are in one directory, whilst the the v2.1 are separated into different files by plugin. There's some rather old documentation on how to generate the api samples themselves (hint: directories aren't made automatically) here: https://blueprints.launchpad.net/nova/+spec/nova-api-samples Personally I wouldn't bother with any xml support if you do decide to support v2 as its deprecated anyway. Hope this helps. Feel free to add me as a reviewer for the api parts of your changesets. Regards, Chris In particular I noticed I had to modify the file nova/nova.egg-info/entry_points.txt to make my v3 api to load, but this file seems not to be under versioning, is this file modified only after the changes are merged? On 11/16/14 23:55, Christopher Yeoh wrote: On Thu, Nov 13, 2014 at 12:14 AM, Pasquale Porreca pasquale.porr...@dektech.com.au wrote: Hello I am working on an api for a new feature in nova, but I am wondering what is the correct way to add a new extension: should it be supported by v2, v3 or both? You need now to have at least a v2.1 (formerly known as v3) extension. V2 support if you want but I think once v2.1 is fully merged and tested (which may not be that far away at all) we should freeze v2 and rely just on v2.1 for new features. Otherwise the interaction between v2.1 being exactly equivalent to v2 plus having microversion support for v2.1 will get a bit confusing. As the other Chris mentioned, the first step however is to get a nova-spec submitted which needs to fully describe the API additions that you want to make. Regards, Chris BR -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] v2 or v3 for new api
On Tue, Nov 18, 2014 at 2:31 AM, Pasquale Porreca pasquale.porr...@dektech.com.au wrote: Thank you very much Christopher On 11/17/14 12:15, Christopher Yeoh wrote: Yes, sorry documentation has been on our todo list for too long. Could I get you to submit a bug report about the lack of developer documentation for api plugins? It might hurry us up :-) I reported as a bug and subscribed you to it. https://bugs.launchpad.net/ nova/+bug/1393455 Thanks! In the meantime, off the top of my head. you'll need to create or modify the following files in a typical plugin: setup.cfg - add an entry in at least the nova.api.v3.extensions section etc/nova/policy.json - an entry for the permissions for you plugin, perhaps one per api method for maximum flexibility. Also will need a discoverable entry (lots of examples in this file) nova/tests/unit/fake_policy.json (similar to policy.json) I wish I had asked about this before, I found yet these files, but I confess it took quite a bit of time to guess I had to modify them (I actually didn't modify yet fake_policy, but my tests are still not completed). What about nova/nova.egg-info/entry_points.txt I mentioned earlier? The entry points file is automatically updated based on setup.cfg nova/api/openstack/compute/plugins/v3/your_plugin.py - please make the alias name something os-scheduler-hints rather than OS-SCH-HNTS. No skimping on vowels. Probably the easiest way at this stage without more doco is look for for a plugin in that directory that does the sort of the thing you want to do. Following the path of other plugins, I created a module nova/api/openstack/compute/plugins/v3/node_uuid.py, while the class is NodeUuid(extensions.V3APIExtensionBase) the alias is os-node-uuid and the actual json parameter is node_uuid. I hope this is correct... nova/tests/unit/nova/api/openstack/compute/contrib/test_your_plugin.py - we have been combining the v2 and v2.1(v3) unittests to share as much as possible, so please do the same here for new tests as the v3 directory will be eventually removed. There's quite a few examples now in that directory of sharing unittests between v2.1 and v2 but with a new extension the customisation between the two should be pretty minimal (just a bit of inheritance to call the right controller) Very good to know. I put my test in nova/tests/unit/api/openstack/plugins/v3 , but I was getting confused by the fact only few tests were in this folder while the tests in nova/tests/unit/api/openstack/compute/contrib/ covered both v2 and v2.1 cases. So should I move my test in nova/tests/unit/api/openstack/compute/contrib/ folder, right? Yes nova/tests/unit/integrated/v3/test_your_plugin.py nova/tests/unit/integrated/test_api_samples.py Sorry the api samples tests are not unified yet. So you'll need to create two. All of the v2 api sample tests are in one directory, whilst the the v2.1 are separated into different files by plugin. There's some rather old documentation on how to generate the api samples themselves (hint: directories aren't made automatically) here: https://blueprints.launchpad.net/nova/+spec/nova-api-samples Personally I wouldn't bother with any xml support if you do decide to support v2 as its deprecated anyway. After reading your answer I understood I have to work more on this part :) Hope this helps. Feel free to add me as a reviewer for the api parts of your changesets. It helps a lot! I will add you for sure as soon as I will upload my code. For now the specification has still to be approved, so I think I have to wait before to upload it, is that correct? This is the blueprint link anyway: https://blueprints.launchpad. net/nova/+spec/use-uuid-v1 So it won't hurt to upload the code before the spec is approved if you're looking for some early feedback, but I'd recommend setting it to Work in Progress otherwise it's likely to get -2'd pending spec approval Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] fix latency on requirements breakage
On Tue, Nov 18, 2014 at 9:32 AM, Sean Dague s...@dague.net wrote: waiting extra long for valid test results. People don't realize their code can't pass and just keep pushing patches up consuming resources which means that parts of the project that could pass tests, is backed up behind 100% guarunteed failing parts. All in all, not a great system. Maybe a MOTD at the top of http://review.openstack.org could help here? Have a button that the QA/infra people can hit when everything is broken that puts up a message there asking people to stop rechecking/submitting patches. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] v2 or v3 for new api
On Thu, Nov 13, 2014 at 12:14 AM, Pasquale Porreca pasquale.porr...@dektech.com.au wrote: Hello I am working on an api for a new feature in nova, but I am wondering what is the correct way to add a new extension: should it be supported by v2, v3 or both? You need now to have at least a v2.1 (formerly known as v3) extension. V2 support if you want but I think once v2.1 is fully merged and tested (which may not be that far away at all) we should freeze v2 and rely just on v2.1 for new features. Otherwise the interaction between v2.1 being exactly equivalent to v2 plus having microversion support for v2.1 will get a bit confusing. As the other Chris mentioned, the first step however is to get a nova-spec submitted which needs to fully describe the API additions that you want to make. Regards, Chris BR -- Pasquale Porreca DEK Technologies Via dei Castelli Romani, 22 00040 Pomezia (Roma) Mobile +39 3394823805 Skype paskporr ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Cross-Project Liaison for the API Working Group
On Sat, Nov 15, 2014 at 7:26 AM, Everett Toews everett.to...@rackspace.com wrote: On Nov 14, 2014, at 1:43 PM, Jay Pipes jaypi...@gmail.com wrote: On 11/14/2014 05:13 PM, Everett Toews wrote: The liaison should be a core reviewer for the project, but does not need to be the PTL. By default, the liaison will be the PTL. ]Anyway, the outcome of the email exchange with Eoghan was that I recommended he submit a core for the API liaison to start, and that I would raise the issue on the ML to see if those conditions may be loosened to include non-cores. And... here is that issue being raised :) I’m totally fine with that. Ideally it’s the person who is the most passionate about the API from the team, regardless of core status. The current wording on the Cross-Project Liaisons page is The liaison should be a core reviewer for the project, but does not need to be the PTL.” I think the key word there is _should_. Naturally, it’s preferable to want a core reviewer in this role because they have more say about what gets into the code base but it’s not an absolute must. We can loosen the wording further but I think it’s okay to proceed with a non-core reviewer, especially if that person is passionate about APIs. My 2c is we should say The liason should be the PTL or whomever they delegate to be their representative and not mention anything about the person needing to be a core developer. It removes any ambiguity about who ultimately decides who the liason is (the PTL) without saying that they have to do the work themselves. Regards, Chris Everett ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [api] API WG Meeting
Hi, The patch setting the time for the API Working Group meeting has merged: https://review.openstack.org/#/c/128332/ The time for the meeting is Thursday UTC In other timezones the meeting is at: EST 20:00 (Wed) Japan 09:00 (Thu) China 08:00 (Thu) ACDT 10:30 (Thu) The meeting details are here: https://wiki.openstack.org/wiki/Meetings/API-WG I know this is very short notice and some people have already started travelling for Summit. But I'll be there in case anyone else turns up. We won't of course have a meeting next week. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API Workgroup git repository
On Thu, 23 Oct 2014 05:53:21 + Shaunak Kashyap shaunak.kash...@rackspace.com wrote: Thanks Chris. And am I correct in assuming that no weekly api-wg meetings are currently going on? I don’t see anything here anyway: https://wiki.openstack.org/wiki/Meetings. They haven't started yet. Not sure if we'll be squeezing one in before summit or not. The proposed meeting time patch only just recently merged (Thursdays UTC ) https://review.openstack.org/#/c/128332/2 so perhaps thats why its not there yet. Chris Shaunak On Oct 22, 2014, at 10:39 PM, Christopher Yeoh cbky...@gmail.commailto:cbky...@gmail.com wrote: On Thu, Oct 23, 2014 at 3:51 PM, Shaunak Kashyap shaunak.kash...@rackspace.commailto:shaunak.kash...@rackspace.com wrote: If I have a proposal to make for a guideline, how do I do it? Do I simply create a gerrit review for a file under http://git.openstack.org/cgit/openstack/api-wg/tree/guidelines? Yes, either additions to the existing files or add new ones if there is nothing currently appropriate. We still need to talk about the exact process for accepting a patch yet, but I was thinking about something along the lines of (and haven't really discussed it with anyone yet): - must be up for at least a week - discussed at a weekly api-wg meeting - no more than one negative vote (ok we might need something to break a deadlock if we really can't get consensus over something but I think there should be a lot of incentive to try very hard to do so). Though for now any setup work in the repository is getting approved by Jay or I if it looks reasonable. Chris Shaunak On Oct 22, 2014, at 3:34 PM, Christopher Yeoh cbky...@gmail.commailto:cbky...@gmail.com wrote: On Wed, 22 Oct 2014 20:36:27 + Everett Toews everett.to...@rackspace.commailto:everett.to...@rackspace.com wrote: I notice at the top of the GitHub mirror page [1] it reads, API Working Group http://openstack.orghttp://openstack.org/” Can we get that changed to API Working Group https://wiki.openstack.org/wiki/API_Working_Group”? That URL would be much more helpful to people who come across the GitHub repo. It's not a code change so we would need a repo owner to actually make the change. Who should I contact about that? I think this will do it: https://review.openstack.org/130377 Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [api] API Workgroup git repository
Hi, The API Workgroup git repository has been setup and you can access it here. http://git.openstack.org/cgit/openstack/api-wg/ There is some content there though not all the proposed guidelines from the wiki page are in yet: https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines Please feel free to start submitting patches to the document. I have submitted a patch to convert the initial content from markdown to rst and setup the tox targets to produce an html document. Seemed to be an easier route as it seems to be the preferred format for OpenStack projects and we can just copy all the build/check bits from the specs repositories. Also doesn't require any changes to required packages. https://review.openstack.org/130120 Until this is merged its probably better to base any patches on this one. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API Workgroup git repository
On Wed, 22 Oct 2014 14:44:26 -0500 Anne Gentle a...@openstack.org wrote: On Wed, Oct 22, 2014 at 2:26 PM, Steve Martinelli steve...@ca.ibm.com wrote: we could set up a job to publish under docs.o.org/api-wg pretty easily - it seems like a good place to start to publish this content. thanks for getting the repo all setup chris and jay. Thanks for setting up the repo. We probably want it to go to docs.openstack.org/developer/api-wg and link to it from here: http://docs.openstack.org/developer/openstack-projects.html That sounds good to me. Steve has a proposed patch we can use here: https://review.openstack.org/#/c/130363/ I'm not sure of the syntax myself (I had a bit of a look into it last night) Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API Workgroup git repository
On Wed, 22 Oct 2014 20:36:27 + Everett Toews everett.to...@rackspace.com wrote: I notice at the top of the GitHub mirror page [1] it reads, API Working Group http://openstack.org” Can we get that changed to API Working Group https://wiki.openstack.org/wiki/API_Working_Group”? That URL would be much more helpful to people who come across the GitHub repo. It's not a code change so we would need a repo owner to actually make the change. Who should I contact about that? I think this will do it: https://review.openstack.org/130377 Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API Workgroup git repository
On Thu, Oct 23, 2014 at 3:51 PM, Shaunak Kashyap shaunak.kash...@rackspace.com wrote: If I have a proposal to make for a guideline, how do I do it? Do I simply create a gerrit review for a file under http://git.openstack.org/cgit/openstack/api-wg/tree/guidelines? Yes, either additions to the existing files or add new ones if there is nothing currently appropriate. We still need to talk about the exact process for accepting a patch yet, but I was thinking about something along the lines of (and haven't really discussed it with anyone yet): - must be up for at least a week - discussed at a weekly api-wg meeting - no more than one negative vote (ok we might need something to break a deadlock if we really can't get consensus over something but I think there should be a lot of incentive to try very hard to do so). Though for now any setup work in the repository is getting approved by Jay or I if it looks reasonable. Chris Shaunak On Oct 22, 2014, at 3:34 PM, Christopher Yeoh cbky...@gmail.com wrote: On Wed, 22 Oct 2014 20:36:27 + Everett Toews everett.to...@rackspace.com wrote: I notice at the top of the GitHub mirror page [1] it reads, API Working Group http://openstack.org” Can we get that changed to API Working Group https://wiki.openstack.org/wiki/API_Working_Group”? That URL would be much more helpful to people who come across the GitHub repo. It's not a code change so we would need a repo owner to actually make the change. Who should I contact about that? I think this will do it: https://review.openstack.org/130377 Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] APIImpact flag for nova specs
On Mon, 20 Oct 2014 14:44:15 -0500 Anne Gentle a...@openstack.org wrote: I think adding APIImpact will be useful. I also want to point to the addition of Compute v2 (haven't yet proposed a spec for v2.1) to the nova-specs repo here: https://review.openstack.org/#/c/129329/ The goal is to move information from the compute-api repo into the -specs repo. I sent an email to the PTLs in August and then added it in the What's Up Doc Oct 7th so hopefully this doesn't take anyone by surprise. You'll notice if you review that I don't have the template test in place that exist for the other blueprint templates, rather the ideal is that a new file would be added into api/v2.1 if a blueprint affects the API design that describes the correct response/request, error codes, and other relevant info. Also if the feature affects faults, limits, links, pagination, and so on, the spec review would address that in the api spec. I think this would be a good thing to do. I've added some comments on the review. Also here is the review for adding a requirement for an APIImpact flag in the commit message: https://review.openstack.org/#/c/129757/ Regards, Chris Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Request Validation - Stoplight
On Mon, 20 Oct 2014 10:38:58 -0400 Jay Pipes jaypi...@gmail.com wrote: For stackers who are interested in different validation frameworks to implement validation, I recommend checking out Stoplight. Just my two cents on this particular topic, I think it's more important to standardize ways in which our public REST APIs expose the payload expectations and response schemas to clients. In other words... we need to focus on methods for API discovery. Once you have standardized resource URI, request payload, and response schema discovery, then any number of validation libraries may be used. I agree standardising our APIs is more important. However there is an advantage to projects using the same libraries where practical. Its easier for developers to move from project to project, fewer dependencies for the project overall and when we do run into problems with libraries there are more people familiar with library. However I don't think we should be mandating specific libraries, but we can make recommendations (good or bad) based on actual experience. This will be especially useful to new projects starting up to benefit from the pain other projects have experienced. Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Request Validation - Stoplight
On Tue, Oct 21, 2014 at 12:15 PM, Kenichi Oomichi oomi...@mxs.nes.nec.co.jp wrote: Hi Amit, Thanks for picking this topic up, Honestly I don't have a strong opinion about validation libraries. Each project implements based on different web frameworks and the options of validation libraries would be limited from its web framework. For example, Nova implements its own wsgi framework and it is difficult to use pecan/wsme due to its API routing/parameter names. So Nova uses jsonschema for a new API(Nova v2.1 API) because of its flexibility and portability. From quick seeing, stoplight seems flexible and it could cover many cases. That would be nice for me, but I'm not sure that stoplight is the best library because jsonschema is a common way/library and portable. Related to this topic, I'd like to suggest that we have common validation patterns across OpenStack projects. Now each project contains its owns validation patterns for the other project's resource. For example, Nova contains validation patterns for project-id and image-id on the code[1]. Ideally, these validation patterns would be nice to be ported/shared from Keystone and Glance and it is the best to use the same validation patterns between whole OpenStack projects for consistent interfaces. Maybe we can implement these patterns even if using different validation libraries. This sounds good. Would you mind adding it to the wiki here? https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines So we don't lose track of it - we don't have the git/gerrit repository up quite yet. Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] APIImpact flag for nova specs
Hi, I was wondering what people thought of having a convention of adding an APIImpact flag to proposed nova specs commit messages where the Nova API will change? It would make it much easier to find proposed specs which affect the API as its not always clear from the gerrit summary listing. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] APIImpact flag for nova specs
On Wed, Oct 15, 2014 at 7:31 PM, Alex Xu x...@linux.vnet.ibm.com wrote: On 2014年10月15日 14:20, Christopher Yeoh wrote: Hi, I was wondering what people thought of having a convention of adding an APIImpact flag to proposed nova specs commit messages where the Nova API will change? It would make it much easier to find proposed specs which affect the API as its not always clear from the gerrit summary listing. +1, and is there any tool can be used by search flag? Can use the message: filter in the gerrit web search interface to search in commit messages, or alternatively use gerritlib to write something custom. Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Forming the API Working Group
On Tue, 14 Oct 2014 09:57:01 -0400 Jay Pipes jaypi...@gmail.com wrote: On 10/14/2014 05:04 AM, Alex Xu wrote: There is one reason to think about what projects *currently* do. When we choice which convention we want. For example, the CamelCase and snake_case, if the most project use snake_case, then choice snake_case style will be the right. I would posit that the reason we have such inconsistencies in our project's APIs is that we haven't taken a stand and said this is the way it must be. Not only have we not taken a stand, but we haven't actually told anyone what they should do in the first place. I don't think its defiance on the part of the developers of the API its just that they don't know and without any guidance just do what they think is reasonable. And a lot of the decisions (eg project vs tenant or instance vs server) are fairly arbitrary - we just need to choose one. There's lots of examples of inconsistencies out in the OpenStack APIs. We can certainly use a wiki or etherpad page to document those inconsistencies. But, eventually, this working group should produce solid decisions that should be enforced across *future* OpenStack APIs. And that guidance should be forthcoming in the next month or so, not in one or two release cycles. I agree. I personally think proposing patches to an openstack-api repository is the most effective way to make those proposals. Etherpads and wiki pages are fine for dumping content, but IMO, we don't need to dump content -- we already have plenty of it. We need to propose guidelines for *new* APIs to follow. ok. I'm happy to go that way - I guess I disagree about us already having a lot of content. I haven't yet seen (maybe I've missed them) API guidelines suggestions from most of the openstack projects. But lets just get started :-) Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Forming the API Working Group
On Tue, 14 Oct 2014 10:29:34 -0500 Lance Bragstad lbrags...@gmail.com wrote: I found a couple of free times available for a weekly meeting if people are interested: https://review.openstack.org/#/c/128332/2 Not sure if a meeting time has been hashed out already or not, and if it has I'll change the patch accordingly. If not, we can iterate on possible meeting times in the review if needed. This was to just get the ball rolling if we want a weekly meeting. I proposed one review for Thursdays at 2100 and a second patch set for 2000, UTC. That can easily change, but those were two times that didn't conflict with the existing meeting schedules in #openstack-meeting. So UTC 2000 is 7am Sydney, 5am Tokyo and 4am Beijing time which is pretty early in the day. I'd suggest UTC if that's not too late for others who'd like to participate. Chris On Tue, Oct 14, 2014 at 3:55 AM, Thierry Carrez thie...@openstack.org wrote: Jay Pipes wrote: On 10/13/2014 07:11 PM, Christopher Yeoh wrote: I guess we could also start fleshing out in the repo how we'll work in practice too (eg once the document is stable what process do we have for making changes - two +2's is probably not adequate for something like this). We can make it work exactly like the openstack/governance repo, where ttx has the only ability to +2/+W approve a patch for merging, and he tallies a majority vote from the TC members, who vote -1 or +1 on a proposed patch. Instead of ttx, though, we can have an API working group lead selected from the set of folks currently listed as committed to the effort? Yes, the working group should select a chair who would fill the same role I do for the TC (organize meetings, push agenda, tally votes on reviews, etc.) That would be very helpful in keeping that diverse group on track. Now you just need some volunteer :) -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Forming the API Working Group
On Tue, 14 Oct 2014 09:45:44 -0400 Jay Pipes jaypi...@gmail.com wrote: On 10/14/2014 12:52 AM, Christopher Yeoh wrote: On Mon, 13 Oct 2014 22:20:32 -0400 Jay Pipes jaypi...@gmail.com wrote: On 10/13/2014 07:11 PM, Christopher Yeoh wrote: On Mon, 13 Oct 2014 10:52:26 -0400 And whilst I don't have a problem with having some guidelines which suggest a future standard for APIs, I don't think we should be requiring any type of feature which has not yet been implemented in at least one, preferably two openstack projects and released and tested for a cycle. Eg standards should be lagging rather than leading. What about features in some of our APIs that are *not* preferable? For instance: API extensions. I think we've seen where API extensions leads us. And it isn't pretty. Would you suggest we document what a Nova API extension or a Neutron API extension looks like and then propose, for instance, not to ever do it again in future APIs and instead use schema discoverability? So if we had standards leading development rather than lagging in the past then API extensions would have ended up in the standard because we once thought they were a good idea. Perhaps we should distinguish in the documentation between recommendations (future looking) and standards (proven it works well for us). The latter would be potentially enforced a lot more strictly than the former. In the case of extensions I think we should have a section documenting why we think they're a bad idea and new projects certainly shouldn't use them. But also give some advice around if they are used what features they should have (eg version numbers!). Given the time that it takes to make major API infrastructure changes it is inevitable that there will be api extensions added in the short to medium term. Because API development will not just stop while API infrastructure is improved. I think it will be better in git (but we also need it in gerrit) when it comes to resolving conflicts and after we've established a decent document (eg when we have more content). I'm just looking to make it as easy as possible for anyone to add any guidelines now. Once we've actually got something to discuss then we use git/gerrit with patches proposed to resolve conflicts within the document. Of course it would be in Gerrit. I just put it up on GitHub first because I can't just add a repo into the openstack/ code namespace... :) I've submitted a patch to add an api-wg project using your repository as the initial content for the git repository. https://review.openstack.org/#/c/128466/ Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Forming the API Working Group
On Mon, 13 Oct 2014 10:52:26 -0400 Jay Pipes jaypi...@gmail.com wrote: On 10/10/2014 02:05 AM, Christopher Yeoh wrote: I agree with what you've written on the wiki page. I think our priority needs to be to flesh out https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines so we have something to reference when reviewing specs. At the moment I see that document as something anyone should be able to document a project's API convention even if they conflict with another project for the moment. Once we've got a fair amount of content we can start as a group resolving any conflicts. Agreed that we should be fleshing out the above wiki page. How would you like us to do that? Should we have an etherpad to discuss individual topics? Having multiple people editing the wiki page offering commentary seems a bit chaotic, and I think we would do well to have the Gerrit review process in place to handle proposed guidelines and rules for APIs. See below for specifics on this... Honestly I don't think we have enough content yet to have much of a discussion. I started the wiki page https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines in the hope that people from other projects would start adding conventions that they use in their projects. I think its fine for the moment if its contradictory, we just need to gather what projects currently do (or want to do) in one place so we can start discussing any contradictions. So I'd again encourage anyone interested in APIs from the various projects to just start dumping their project viewpoint in there. Speaking of the wiki page, I wrote it very matter-of-factly. As if this is the way things are. They’re not. The wiki page is just a starting point. If something was missed, add it. If something can be improved, improve it. Let’s try to keep it simple though. One problem with API WG members reviewing spec proposals that affect the API is finding the specs in the first place across many different projects repositories. I've said for a while now that I would love to have separate repositories -- ala the Keystone API in the openstack/identity-api repository -- that contains specifications for APIs in a single format (APIBlueprint was suggested at one point, but Swagger 2.0 seems to me to have more upside). I also think it would be ideal to have an openstack/openstack-api repo that would house guidelines and rules that this working group came up with, along with examples of appropriate usage. This repo would function very similar to the openstack/governance [1] repo that the TC uses to flesh out proposals on community, release management, and governance changes. If people are OK with this idea, I will go ahead and create the repo and add the wiki page content as the initial commit, then everyone can simply submit patches to the document(s) using the normal Gerrit process, and we can iterate on these things using the same tools as other repositories. I like the idea of a repo and using Gerrit for discussions to resolve issues. I don't think it works so well when people are wanting to dump lots of information in initially. Unless we agree to just merge anything vaguely reasonable and then resolve the conflicts later when we have a reasonable amount of content. Otherwise stuff will get lost in gerrit history comments and people's updates to the document will overwrite each other. I guess we could also start fleshing out in the repo how we'll work in practice too (eg once the document is stable what process do we have for making changes - two +2's is probably not adequate for something like this). Regards, Chris Best, -jay [1] https://review.openstack.org/#/q/status:open+project:openstack/governance,n,z I invite everyone who chimed in on the original thread [1] that kicked this off to add themselves as a member committed to making the OpenStack APIs better. I’ve Cc’d everyone who asked to be kept in the loop. I already see some cross project summit topics [2] on APIs. But frankly, with the number of people committed to this topic, I’d expect there to be more. I encourage everyone to submit more API related sessions with better descriptions and goals about what you want to achieve in those sessions. Yea if there is enough content in the API guidelines then perhaps some time can be spent on working on resolving any conflicts in the document so projects know what direction to head in. Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Forming the API Working Group
On Mon, 13 Oct 2014 22:20:32 -0400 Jay Pipes jaypi...@gmail.com wrote: On 10/13/2014 07:11 PM, Christopher Yeoh wrote: On Mon, 13 Oct 2014 10:52:26 -0400 Jay Pipes jaypi...@gmail.com wrote: On 10/10/2014 02:05 AM, Christopher Yeoh wrote: I agree with what you've written on the wiki page. I think our priority needs to be to flesh out https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines so we have something to reference when reviewing specs. At the moment I see that document as something anyone should be able to document a project's API convention even if they conflict with another project for the moment. Once we've got a fair amount of content we can start as a group resolving any conflicts. Agreed that we should be fleshing out the above wiki page. How would you like us to do that? Should we have an etherpad to discuss individual topics? Having multiple people editing the wiki page offering commentary seems a bit chaotic, and I think we would do well to have the Gerrit review process in place to handle proposed guidelines and rules for APIs. See below for specifics on this... Honestly I don't think we have enough content yet to have much of a discussion. I started the wiki page https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines in the hope that people from other projects would start adding conventions that they use in their projects. I think its fine for the moment if its contradictory, we just need to gather what projects currently do (or want to do) in one place so we can start discussing any contradictions. Actually, I don't care all that much about what projects *currently* do. I want this API working group to come up with concrete guidelines and rules/examples of what APIs *should* look like. What projects currently do gives us a baseline to work from. It also should expose where we have currently have inconsistencies between projects. And whilst I don't have a problem with having some guidelines which suggest a future standard for APIs, I don't think we should be requiring any type of feature which has not yet been implemented in at least one, preferably two openstack projects and released and tested for a cycle. Eg standards should be lagging rather than leading. So I'd again encourage anyone interested in APIs from the various projects to just start dumping their project viewpoint in there. I went ahead and just created a repository that contained all the stuff that should be pretty much agreed-to, and a bunch of stub topic documents that can be used to propose specific ideas (and get feedback on) here: http://github.com/jaypipes/openstack-api Hopefully, you can give it a look and get a feel for why I think the code review process will be better than the wiki for controlling the deliverables produced by this team... I think it will be better in git (but we also need it in gerrit) when it comes to resolving conflicts and after we've established a decent document (eg when we have more content). I'm just looking to make it as easy as possible for anyone to add any guidelines now. Once we've actually got something to discuss then we use git/gerrit with patches proposed to resolve conflicts within the document. I like the idea of a repo and using Gerrit for discussions to resolve issues. I don't think it works so well when people are wanting to dump lots of information in initially. Unless we agree to just merge anything vaguely reasonable and then resolve the conflicts later when we have a reasonable amount of content. Otherwise stuff will get lost in gerrit history comments and people's updates to the document will overwrite each other. I guess we could also start fleshing out in the repo how we'll work in practice too (eg once the document is stable what process do we have for making changes - two +2's is probably not adequate for something like this). We can make it work exactly like the openstack/governance repo, where ttx has the only ability to +2/+W approve a patch for merging, and he tallies a majority vote from the TC members, who vote -1 or +1 on a proposed patch. Instead of ttx, though, we can have an API working group lead selected from the set of folks currently listed as committed to the effort? Yep, that sounds fine, though I don't think a simple majority is sufficient for something like api standards. We either get consensus or we don't include it in the final document. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Forming the API Working Group
Hi Everett, Great to see things moving with the API Working Group! On Thu, Oct 9, 2014 at 9:35 AM, Everett Toews everett.to...@rackspace.com wrote: https://wiki.openstack.org/wiki/API_Working_Group This is the start of the API Working Group (API WG). To avoid bike shedding over the name of the working group, I decided to title the wiki page API Working Group. Simple, to the point, and avoids loaded terms like standards, best practices, guidelines, conventions, etc. The point isn’t what we name it. The point is what action we take about it. I propose the deliverables in the API WG wiki page. I agree with what you've written on the wiki page. I think our priority needs to be to flesh out https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines so we have something to reference when reviewing specs. At the moment I see that document as something anyone should be able to document a project's API convention even if they conflict with another project for the moment. Once we've got a fair amount of content we can start as a group resolving any conflicts. Speaking of the wiki page, I wrote it very matter-of-factly. As if this is the way things are. They’re not. The wiki page is just a starting point. If something was missed, add it. If something can be improved, improve it. Let’s try to keep it simple though. One problem with API WG members reviewing spec proposals that affect the API is finding the specs in the first place across many different projects repositories. I invite everyone who chimed in on the original thread [1] that kicked this off to add themselves as a member committed to making the OpenStack APIs better. I’ve Cc’d everyone who asked to be kept in the loop. I already see some cross project summit topics [2] on APIs. But frankly, with the number of people committed to this topic, I’d expect there to be more. I encourage everyone to submit more API related sessions with better descriptions and goals about what you want to achieve in those sessions. Yea if there is enough content in the API guidelines then perhaps some time can be spent on working on resolving any conflicts in the document so projects know what direction to head in. Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Nova API meeting
Hi, Just a reminder that the weekly Nova API meeting is being held tomorrow Friday UTC . We encourage cloud operators and those who use the REST API such as SDK developers and others who and are interested in the future of the API to participate. In other timezones the meeting is at: EST 20:00 (Thu) Japan 09:00 (Fri) China 08:00 (Fri) ACDT 9:30 (Fri) The proposed agenda and meeting details are here: https://wiki.openstack.org/wiki/Meetings/NovaAPI Please feel free to add items to the agenda. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs][tc] How to scale Documentation
On Mon, 06 Oct 2014 11:24:30 +0200 Julien Danjou jul...@danjou.info wrote: On Sun, Oct 05 2014, Joshua Harlow wrote: I agree. I think we should have documentation be part of our policy to accept patches, just as we do with e.g. unit testing. Forcing people to write documentation along the patch will help writing better code, having a better understanding of what the patch does for reviewers, etc… Agreed. And a lot of the confusion we get between what is documented to happen and what is implemented in the code is due to different people writing the documentation versus writing the code. And there has been insufficient information (other than the code) left for people to write accurate documentation. For API docs I wonder if we should bringing more of that into the project tree so we can enforce updating of documentation when the code gets updated. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] No Nova API meeting this week
Hi, About half the regulars at the Nova API meeting are on vacation this week so I'm cancelling this weeks meeting as I don't think there is anything urgent to discuss. We'll meet as usual next week. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] why do we have os-attach-interfaces in the v3 API?
On Thu, 02 Oct 2014 15:57:55 -0500 Matt Riedemann mrie...@linux.vnet.ibm.com wrote: The os-interface (v2) and os-attach-interfaces (v3) APIs are only used for the neutron network API, you'll get a NotImplemented if trying to call the related methods with nova-network [1]. Since we aren't proxying to neutron in the v3 API (v2.1), why does os-attach-interfaces [2] exist? Was this just an oversight? If so, please allow me to delete it. :) The proxying work was not done in Juno due to time constraints, but I think we should be able to cover it early in Kilo (most of the patches are pretty much ready). To have a V2.1 which is functionality equivalent to V2 (allowing us to remove the V2 code) we have to implement proxying. Chris [1] http://git.openstack.org/cgit/openstack/nova/tree/nova/network/api.py?id=2014.2.rc1#n310 [2] http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/plugins/v3/attach_interfaces.py?id=2014.2.rc1 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Kilo Blueprints and Specs
On Mon, 29 Sep 2014 13:32:57 -0700 Joe Gordon joe.gord...@gmail.com wrote: On Mon, Sep 29, 2014 at 5:23 AM, Gary Kotton gkot...@vmware.com wrote: Hi, Is the process documented anywhere? That is, if say for example I had a spec approved in J and its code did not land, how do we go about kicking the tires for K on that spec. Specs will need be re-submitted once we open up the specs repo for Kilo. The Kilo template will be changing a little bit, so specs will need a little bit of reworking. But I expect the process to approve previously approved specs to be quicker Am biased given I have a spec approved for Juno which we didn't quite fully merge which we want to finish off early in Kilo (most of the patches are very close already to being ready to merge), but I think we should give priority to reviewing specs already approved in Juno and perhaps only require one +2 for re-approval. Otherwise we'll end up wasting weeks of development time just when there is lots of review bandwidth available and the CI system is lightly loaded. Honestly, ideally I'd like to just start merging as soon as Kilo opens. Nothing has changed between Juno FF and Kilo opening so there's really no reason that an approved Juno spec should not be reapproved. Chris Thanks Gary On 9/29/14, 1:07 PM, John Garbutt j...@johngarbutt.com wrote: On 27 September 2014 00:31, Joe Gordon joe.gord...@gmail.com wrote: On Thu, Sep 25, 2014 at 9:21 AM, John Garbutt j...@johngarbutt.com wrote: On 25 September 2014 14:10, Daniel P. Berrange berra...@redhat.com wrote: The proposal is to keep kilo-1, kilo-2 much the same as juno. Except, we work harder on getting people to buy into the priorities that are set, and actively provoke more debate on their correctness, and we reduce the bar for what needs a blueprint. We can't have 50 high priority blueprints, it doesn't mean anything, right? We need to trim the list down to a manageable number, based on the agreed project priorities. Thats all I mean by slots / runway at this point. I would suggest we don't try to rank high/medium/low as that is too coarse, but rather just an ordered priority list. Then you would not be in the situation of having 50 high blueprints. We would instead naturally just start at the highest priority and work downwards. OK. I guess I was fixating about fitting things into launchpad. I guess having both might be what happens. The runways idea is just going to make me less efficient at reviewing. So I'm very much against it as an idea. This proposal is different to the runways idea, although it certainly borrows aspects of it. I just don't understand how this proposal has all the same issues? The key to the kilo-3 proposal, is about getting better at saying no, this blueprint isn't very likely to make kilo. If we focus on a smaller number of blueprints to review, we should be able to get a greater percentage of those fully completed. I am just using slots/runway-like ideas to help pick the high priority blueprints we should concentrate on, during that final milestone. Rather than keeping the distraction of 15 or so low priority blueprints, with those poor submitters jamming up the check queue, and constantly rebasing, and having to deal with the odd stray review comment they might get lucky enough to get. Maybe you think this bit is overkill, and thats fine. But I still think we need a way to stop wasting so much of peoples time on things that will not make it. The high priority blueprints are going to end up being mostly the big scope changes which take alot of time to review probably go through many iterations. The low priority blueprints are going to end up being the small things that don't consume significant resource to review and are easy to deal with in the time we're waiting for the big items to go through rebases or whatever. So what I don't like about the runways slots idea is that removes the ability to be agile and take the initiative to review approve the low priority stuff that would otherwise never make it through. The idea is more around concentrating on the *same* list of things. Certainly we need to avoid the priority inversion of concentrating only on the big things. Its also why I suggested that for kilo-1 and kilo-2, we allow any blueprint to merge, and only restrict it to a specific list in kilo-3, the idea being to maximise the number of things that get completed, rather than merging some half blueprints, but not getting to the good bits. Do we have to decide this now, or can we see how project priorities go and reevaluate half way through Kilo-2? What we need to decide is
Re: [openstack-dev] [oslo] Fate of xmlutils
On Mon, 29 Sep 2014 18:03:20 +0200 Julien Danjou jul...@danjou.info wrote: It seems that Python fixed that issue with 2 modules released on PyPI: https://pypi.python.org/pypi/defusedxml https://pypi.python.org/pypi/defusedexpat I'm no XML expert, and I've only a shallow understanding of the issue, but I wonder if we should put some efforts to drop xmlutils and our custom XML fixes to used instead these 2 modules. Nova XML API support is marked as deprecated in Juno. So hopefully we'll be able to just drop XML and associated helper modules within a couple of cycles. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [NOVA] security group fails to attach to an instance if port-id is specified during boot.
On Fri, 26 Sep 2014 11:25:49 +0400 Oleg Bondarev obonda...@mirantis.com wrote: On Fri, Sep 26, 2014 at 3:30 AM, Day, Phil philip@hp.com wrote: I think the expectation is that if a user is already interaction with Neutron to create ports then they should do the security group assignment in Neutron as well. Agree. However what do you think a user expects when he/she boots a vm (no matter providing port_id or just net_id) and specifies security_groups? I think the expectation should be that instance will become a member of the specified groups. Ignoring security_groups parameter in case port is provided (as it is now) seems completely unfair to me. One option would be to return a 400 if both port id and security_groups is supplied. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Nova API meeting
Hi, Just a reminder that the weekly Nova API meeting is being held tomorrow Friday UTC . We encourage cloud operators and those who use the REST API such as SDK developers and others who and are interested in the future of the API to participate. In other timezones the meeting is at: EST 20:00 (Thu) Japan 09:00 (Fri) China 08:00 (Fri) ACDT 9:30 (Fri) The proposed agenda and meeting details are here: https://wiki.openstack.org/wiki/Meetings/NovaAPI Please feel free to add items to the agenda. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] logging around olso lockutils
On Thu, 25 Sep 2014 08:49:12 -0400 Sean Dague s...@dague.net wrote: #1 - tried to get a lock, but someone else has it. Then we know we've got lock contention. . #2 - something is still holding a lock after some long amount of time. +1 to both. #2 turned out to be a critical bit in understanding one of the worst recent gate impacting issues. You can write a tool today that analyzes the logs and shows you these things. However, I wonder if we could actually do something creative in the code itself to do this already. I'm curious if the creative use of Timers might let us emit log messages under the conditions above (someone with better understanding of python internals needs to speak up here). Maybe it's too much overhead, but I think it's worth at least asking the question. Even a simple log at the end when its finally released if it has been held for a long time would be handy. As matching up acquire/release by eye is not easy. I don't think we get a log message when an acquire is attempted but fails. That might help get a measure of lock contention? The same issue exists when it comes to processutils I think, warning that a command is still running after 10s might be really handy, because it turns out that issue #2 was caused by this, and it took quite a bit of decoding to figure that out. Also I think that perhaps a log message when a period task takes longer than the interval that the task is meant to be run would be a useful warning sign that something odd is going on. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Some ideas for micro-version implementation
On Wed, 24 Sep 2014 09:04:51 -0400 Jay Pipes jaypi...@gmail.com wrote: On 09/24/2014 05:26 AM, Kenichi Oomichi wrote: -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Wednesday, September 24, 2014 12:47 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Nova] Some ideas for micro-version implementation On 09/22/2014 04:27 PM, Brant Knudson wrote: On Fri, Sep 19, 2014 at 1:39 AM, Alex Xu x...@linux.vnet.ibm.com mailto:x...@linux.vnet.ibm.com wrote: Close to Kilo, it is time to think about what's next for nova API. In Kilo, we will continue develop the important feature micro-version. In previous v2 on v3 propose, it's include some implementations can be used for micro-version. (https://review.openstack.org/__#/c/84695/19/specs/juno/v2-on-__v3-api.rst https://review.openstack.org/#/c/84695/19/specs/juno/v2-on-v3-api.rst) But finally, those implementations was considered too complex. So I'm try to find out more simple implementation and solution for micro-version. I wrote down some ideas as blog post at: http://soulxu.github.io/blog/__2014/09/12/one-option-for-__nova-api/ http://soulxu.github.io/blog/2014/09/12/one-option-for-nova-api/ And for those ideas also already done some POC, you can find out in the blog post. As discussion in the Nova API meeting, we want to bring it up to mail-list to discussion. Hope we can get more idea and option from all developers. We will appreciate for any comment and suggestion! Thanks Alex Did you consider JSON Home[1] for this? For Juno we've got JSON Home support in Keystone for Identity v3 (Zaqar was using it already). We weren't planning to use it for microversioning since we weren't planning on doing microversioning, but I think JSON Home could be used for this purpose. Using JSON Home, you'd have relationships that include the version, then the client can check the JSON Home document to see if the server has support for the relationship the client wants to use. [1] http://tools.ietf.org/html/draft-nottingham-json-home-03 ++ I used JSON-Home extensively in the Compute API blueprint I put together a few months ago: http://docs.oscomputevnext.apiary.io/ vNext seems an interesting idea, I thought the implementation way for Nova a little. API Route Discoverability is a nice design, but a root / URL will conflict on current list versions API. Maybe there would be a workaround. Completely agreed, Ken'ichi. The root URL that returns the JSON-Home doc in the vNext API is actually *after* the version in the URI, though... So, the JSON-Home doc would be returned from: http://compute.example.com/vNext/ Of course, replacing vNext with v4 or v42 or whatever the next major version of the API would be. The real root would still return the versions list as it exists today, with a 302 Multiple Choice. Why have a version in the url at all? With microversions it doesn't really make any sense and would just cause more churn for app developers. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [All] API standards working group
On Wed, 24 Sep 2014 16:48:43 + Everett Toews everett.to...@rackspace.com wrote: On Sep 24, 2014, at 9:42 AM, Dean Troyer dtro...@gmail.com wrote: I'll bring an API consumer's perspective. +1 I’d bring an API consumer’s perspective as well. Looks like there’s lots of support for an API WG. What’s the next step? We have the start of some guidelines here: https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines I think it would be very useful if those who are interested in helping out add info to that wiki from the perspective of the project they work on and comment (inline) where what is already there is inconsistent with what they do. User input too as to what they want would be useful. That would give us something much more concrete to discuss and debate rather than just a common desire to have consistency across our APIs. Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [All] API standards working group
On Wed, 24 Sep 2014 13:05:09 -0400 Jay Pipes jaypi...@gmail.com wrote: On 09/24/2014 12:48 PM, Everett Toews wrote: On Sep 24, 2014, at 9:42 AM, Dean Troyer dtro...@gmail.com wrote: I'll bring an API consumer's perspective. +1 I’d bring an API consumer’s perspective as well. Looks like there’s lots of support for an API WG. What’s the next step? Form a WG under the User Committee [1] or is there something more appropriate? Thanks, Everett [1] https://wiki.openstack.org/wiki/Governance/Foundation/UserCommittee#Working_Groups Whatever it ends up being, it needs to have some teeth to it. Otherwise, we're going to end up in the exact same place we're in now, where each project does something slightly different. I think its more an issue around actually having guidelines and educating people where to find them than needing teeth to enforce them. Where I've seen variation in the APIs in Nova its been primarily because people haven't known what the conventions are than anything else. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Some ideas for micro-version implementation
On Mon, 22 Sep 2014 09:29:26 + Kenichi Oomichi oomi...@mxs.nes.nec.co.jp wrote: Before discussing how to implement, I'd like to consider what we should implement. IIUC, the purpose of v3 API is to make consistent API with the backwards incompatible changes. Through huge discussion in Juno cycle, we knew that backwards incompatible changes of REST API would be huge pain against clients and we should avoid such changes as possible. If new APIs which are consistent in Nova API only are inconsistent for whole OpenStack projects, maybe we need to change them again for whole OpenStack consistency. So I think there's three different aspects to microversions which we can consider quit separately: - The format of the client header and what the version number means. Eg is the version number of the format X.Y.Z, what do we increment when we make a bug fix, what do we increment when we make a backwards compatible change and what do we increment when we make backwards incompatible change. Also how does a client request experimental APIs (I believe we have consensus that we really need this to avoid backwards incompatible changes as much as possible as it allows more testing before guaranteeing backwards compatibility) I believe that we can consider this part separately from the next two issues. - The implementation on the nova api side. Eg how do we cleanly handle supporting multiple versions of the api based on the client header (or lack of it which will indicate v2 compatibility. I'll respond directly on Alex's original post - What we are going to use the microversions API feature to do. I think they fall under a few broad categories: - Backwards compatible changes. We desperately need a mechanism that allows us to make backwards compatible changes (eg add another parameter to a response) without having to add another dummy extension. - Significant backwards incompatible changes. The Tasks API and server diagnostics API are probably the best examples of this. - V3 like backwards incompatible changes (consistency fixes). I think getting consensus over backwards compatible changes will be straightforward. However given the previous v2/v3 discussions I don't think we will be able to get consensus over doing all or most of the consistency type fixes even using microversions in the short term. Because with microversions you get all the changes applied before the version that you choose. So from a client application point of view its just as much work as V2 to V3 API transition. I don't think that means we need to put all of these consistency changes off forever though. We need to make backwards incompatible changes in order to implement the Tasks API and new server diagnostics api the way we want to. The Tasks API will eventually cover quite a few interfaces and while say breaking backwards compatibility with the create server api, we can also fix consistency issues in that api at the same time. Clients will need to make changes to their app anyway if they want to take advantage of the new features (or they can just continue to use the old non-tasks enabled API). So as we slowly make backwards incompatible changes to the API for other reasons we can also fix up other issues. Other consistency fixes we can propose on a case by case basis and the user community can have input as to whether the cost (app rework) is worth it without getting a new feature at the same time. But I think its clear that we *need* the microversions mechanism. So we don't need to decide beforehand exactly what we're going to use it for first. I think think its more important that we get a nova-spec approved for the the first two parts - what it looks like from the client point of view. And how we're going to implement it. Regards, Chris For avoiding such situation, I think we need to define what is consistent REST API across projects. According to Alex's blog, The topics might be - Input/Output attribute names - Resource names - Status code The following are hints for making consistent APIs from Nova v3 API experience, I'd like to know whether they are the best for API consistency. (1) Input/Output attribute names (1.1) These names should be snake_case. eg: imageRef - image_ref, flavorRef - flavor_ref, hostId - host_id (1.2) These names should contain extension names if they are provided in case of some extension loading. eg: security_groups - os-security-groups:security_groups config_drive - os-config-drive:config_drive (1.3) Extension names should consist of hyphens and low chars. eg: OS-EXT-AZ:availability_zone - os-extended-availability-zone:availability_zone OS-EXT-STS:task_state - os-extended-status:task_state (1.4) Extension names should contain the prefix os- if the extension is not core. eg: rxtx_factor - os-flavor-rxtx:rxtx_factor os-flavor-access:is_public - flavor-access:is_public (flavor-access extension became core)
Re: [openstack-dev] [Nova] Some ideas for micro-version implementation
On Mon, 22 Sep 2014 09:47:50 -0500 Anne Gentle a...@openstack.org wrote: (1) Input/Output attribute names (1.1) These names should be snake_case. eg: imageRef - image_ref, flavorRef - flavor_ref, hostId - host_id (1.2) These names should contain extension names if they are provided in case of some extension loading. eg: security_groups - os-security-groups:security_groups config_drive - os-config-drive:config_drive Do you mean that the os- prefix should be dropped? Or that it should be maintained and added as needed? Originally (I think) the os- prefix was added to avoid potential name clashes between extensions. Presumably out of tree extensions too as in-tree ones would be picked up pretty quickly. For a while now I've been thinking that perhaps we should just drop the os- prefix. We're trying to discourage optional extensions anyway and I suspect that the pain of maintaining consistency with os- prefixes is not worth the benefit. Where if someone wants to maintain an out of tree plugin they can bear the burden of avoiding name clashes. This would make it much easier for both us (nearly no code changes required) and users (no app changes) when we move some functionality into (or out of) core. (1.3) Extension names should consist of hyphens and low chars. eg: OS-EXT-AZ:availability_zone - os-extended-availability-zone:availability_zone OS-EXT-STS:task_state - os-extended-status:task_state Yes, I don't like the shoutyness of the ALL CAPS. +1! No to ALL CAPS and contractions or disemvowling of words in order to save a few bytes. (1.4) Extension names should contain the prefix os- if the extension is not core. eg: rxtx_factor - os-flavor-rxtx:rxtx_factor os-flavor-access:is_public - flavor-access:is_public (flavor-access extension became core) Do we have a list of core yet? We do sort of have a candidate listed hard coded (its a pretty small list): API_V3_CORE_EXTENSIONS = set(['consoles', 'extensions', 'flavor-access', 'flavor-extra-specs', 'flavor-manage', 'flavors', 'ips', 'os-keypairs', 'server-metadata', 'servers', 'versions']) (3) Status code (3.1) Return 201(Created) if a resource creation/updating finishes before returning a response. eg: create a keypair API: 200 - 201 create an agent API: 200 - 201 create an aggregate API: 200 - 201 Do you mean a 200 becomes a 201? That's a huge doc impact and SDK impact, is it worthwhile? If we do this change, the sooner the better, right? Ideally I think we'd want to do it when breaking a specific part of the API anyway (for say some new feature). Otherwise its something I think we should bring up on a case by case with users and operators. Trade off between returning misleading status codes versus requiring application side changes (potentially, some may just look for 2xx anyway). The TC had an action item a while back (a few months) to start an API style guide. This seems like a good start. Once the questions are discussed I'll get a draft going on the wiki. So back during the Juno design summit at the cross project API consistency session we talked about putting together a wiki page with guidelines for all OpenStack projects. But I think everyone got busy so not much at all happened. I've had a bit of time recently and tried to pull together info from the session as well as some other sources and the start of it is here: https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines So perhaps we could merge what is above into this wiki? It is however still rather Nova specific and we need input from other projects (I'll send out another email specifically about this). Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Some ideas for micro-version implementation
On Mon, 22 Sep 2014 15:27:47 -0500 Brant Knudson b...@acm.org wrote: Did you consider JSON Home[1] for this? For Juno we've got JSON Home support in Keystone for Identity v3 (Zaqar was using it already). We weren't planning to use it for microversioning since we weren't planning on doing microversioning, but I think JSON Home could be used for this purpose. Using JSON Home, you'd have relationships that include the version, then the client can check the JSON Home document to see if the server has support for the relationship the client wants to use. JSON Home is on our list of things to do for the API (but the priority is lower than microversions). We still need microversions anyway because we want to be able to support multiple versions of API through the same resources (ie no url changes). To make life as easy as possible for users so they are not forced to upgrade their app whenever we make backwards incompatible changes. I don't see us supporting every version forever, but it gives them some time to upgrade. Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] REST API style guide (Re: [Nova] Some ideas for micro-version implementation)
On Tue, 23 Sep 2014 09:00:26 +0900 Ken'ichi Ohmichi ken1ohmi...@gmail.com wrote: \ So how about just using HTTP 200(OK) only for status codes? That would give up providing accurate internal status to clients but backwards backwards incompatibilities never happen. No I think that we should where possible return the most accurate status code. A 202 versus 200 is an important distinction for a user of the API (eg do they need to poll for request completion?). How fast we can get to accurate status codes through the API is a different matter though. and I have one more idea for making API consistency of whole OpenStack projects. That is each rule of the style guide is implemented in Tempest. Tempest has its own REST clients for many projects and we can customize them for improving qualities. After defining the REST API style guide, we can add each rule to Tempest's base client class and apply it for all REST APIs which are tested by Tempest. We can keep consistent API for the existing projects and apply the style guide to new projects also by this framework. That's an interesting idea! However we would have a long exception list for a quite a while I think as it was made pretty clear to use we can't make a large number of backwards compatible API changes over a short period of time. (Like a v2-v3 transition was). Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [All] API standards working group
On Tue, 23 Sep 2014 18:18:56 -0400 Jay Pipes jaypi...@gmail.com wrote: On 09/23/2014 05:03 PM, Rochelle.RochelleGrober wrote: jaypi...@gmail.com mailto:jaypi...@gmail.com on Tuesday, September 23, 2014 9:09 AM wrote: _Snip I'd like to say finally that I think there should be an OpenStack API working group whose job it is to both pull together a set of OpenStack API practices as well as evaluate new REST APIs proposed in the OpenStack ecosystem to provide guidance to new projects or new subprojects wishing to add resources to an existing REST API. Best, -jay */[Rocky Grober] /*++ */Jay, are you volunteering to head up the working group? Or at least be an active member? I’ll certainly follow with interest, but I think I have my hands full with the log rationalization working group./* Yes, I'd be willing to head up the working group... or at least participate in it. I'd be very interested in participating as well. At the last Design Summit at the cross project API consistency session we talked about setting up a wiki page for guidelines. I've made a start on this though its very much from a Nova perspective and could do with input from all the other projects. https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Nova API meeting
Hi, Just a reminder that the weekly Nova API meeting is being held tomorrow Friday UTC . We encourage cloud operators and those who use the REST API such as SDK developers and others who and are interested in the future of the API to participate. In other timezones the meeting is at: EST 20:00 (Thu) Japan 09:00 (Fri) China 08:00 (Fri) ACDT 9:30 (Fri) The proposed agenda and meeting details are here: https://wiki.openstack.org/wiki/Meetings/NovaAPI Please feel free to add items to the agenda. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Expand resource name allowed characters
On Sat, 13 Sep 2014 06:48:19 -0400 Sean Dague s...@dague.net wrote: On 09/13/2014 02:28 AM, Kenichi Oomichi wrote: Hi Chris, Thanks for bring it up here, -Original Message- From: Chris St. Pierre [mailto:stpie...@metacloud.com] Sent: Saturday, September 13, 2014 2:53 AM To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [nova] Expand resource name allowed characters We have proposed that the allowed characters for all resource names in Nova (flavors, aggregates, etc.) be expanded to all printable unicode characters and horizontal spaces: https://review.openstack.org/#/c/119741 Currently, the only allowed characters in most resource names are alphanumeric, space, and [.-_]. We have proposed this change for two principal reasons: 1. We have customers who have migrated data forward since Essex, when no restrictions were in place, and thus have characters in resource names that are disallowed in the current version of OpenStack. This is only likely to be useful to people migrating from Essex or earlier, since the current restrictions were added in Folsom. 2. It's pretty much always a bad idea to add unnecessary restrictions without a good reason. While we don't have an immediate need to use, for example, the ever-useful http://codepoints.net/U+1F4A9 in a flavor name, it's hard to come up with a reason people *shouldn't* be allowed to use it. That said, apparently people have had a need to not be allowed to use some characters, but it's not clear why: https://bugs.launchpad.net/nova/+bug/977187 So I guess if anyone knows any reason why these printable characters should not be joined in holy resource naming, speak now or forever hold your peace. I also could not find the reason of current restriction on the bug report, and I'd like to know it as the history. On v2 API(not v2.1), each resource name contains the following restriction for its name: Resource | Length | Pattern ---+-+-- aggregate | 1-255 | nothing backup| nothing | nothing flavor| 1-255 | '^[a-zA-Z0-9. _-]*[a-zA-Z0-9_-]+ | | [a-zA-Z0-9. _-]*$' keypair | 1-255 | '^[a-zA-Z0-9 _-]+$' server| 1-255 | nothing cell | 1-255 | don't contain . and ! On v2.1 API, we have applied the same restriction rule[1] for whole resource names for API consistency, so maybe we need to consider this topic for whole names. [1]: https://github.com/openstack/nova/blob/master/nova/api/validation/parameter_types.py#L44 Honestly, I bet this had to do with how the database used to be set up. So it turns out that utf8 support in MySQL does not support UTF-8 4 byte multibyte characters (only 1-3 bytes). For example if you do a create image call with an image name to glance with a 4 byte multibyte character in the name it will 500. I'd guess we do something similar in places with the Nova API where we have inadequate input validation. If you want 4 byte support you need to use utf8mb4 instead. I don't know if postgresql has any restrictions (I don't think it does) or if db2 does too. But I don't think we can/should make it a complete free for all. It should at most be what most databases support. I think its a big enough change that this late in the cycle we should push it off to Kilo. It's always much easier to loosen input validation than tighten it (or have to have an oops revert on an officially released Nova). Perhaps some tests to verify that everything we allow past the input validation checks we can actually store. That being said, i'm pro letting names be 'utf8'. The id fields that are strings (like flavor_id) I think we should keep constrained, as we do actually do joins on them. (And as jay said, the current utf8 schema is actually highly inefficient and bloaty.) Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Expand resource name allowed characters
On Thu, 18 Sep 2014 13:45:42 +0200 Flavio Percoco fla...@redhat.com wrote: On 09/18/2014 01:28 PM, Sean Dague wrote: On 09/18/2014 07:19 AM, Ken'ichi Ohmichi wrote: 2014-09-18 19:57 GMT+09:00 Sean Dague s...@dague.net: On 09/18/2014 06:38 AM, Christopher Yeoh wrote: On Sat, 13 Sep 2014 06:48:19 -0400 Sean Dague s...@dague.net wrote: We have proposed that the allowed characters for all resource names in Nova (flavors, aggregates, etc.) be expanded to all printable unicode characters and horizontal spaces: https://review.openstack.org/#/c/119741 Currently, the only allowed characters in most resource names are alphanumeric, space, and [.-_]. We have proposed this change for two principal reasons: 1. We have customers who have migrated data forward since Essex, when no restrictions were in place, and thus have characters in resource names that are disallowed in the current version of OpenStack. This is only likely to be useful to people migrating from Essex or earlier, since the current restrictions were added in Folsom. 2. It's pretty much always a bad idea to add unnecessary restrictions without a good reason. While we don't have an immediate need to use, for example, the ever-useful http://codepoints.net/U+1F4A9 in a flavor name, it's hard to come up with a reason people *shouldn't* be allowed to use it. That said, apparently people have had a need to not be allowed to use some characters, but it's not clear why: https://bugs.launchpad.net/nova/+bug/977187 So I guess if anyone knows any reason why these printable characters should not be joined in holy resource naming, speak now or forever hold your peace. I also could not find the reason of current restriction on the bug report, and I'd like to know it as the history. On v2 API(not v2.1), each resource name contains the following restriction for its name: Resource | Length | Pattern ---+-+-- aggregate | 1-255 | nothing backup| nothing | nothing flavor| 1-255 | '^[a-zA-Z0-9. _-]*[a-zA-Z0-9_-]+ | | [a-zA-Z0-9. _-]*$' keypair | 1-255 | '^[a-zA-Z0-9 _-]+$' server| 1-255 | nothing cell | 1-255 | don't contain . and ! On v2.1 API, we have applied the same restriction rule[1] for whole resource names for API consistency, so maybe we need to consider this topic for whole names. [1]: https://github.com/openstack/nova/blob/master/nova/api/validation/parameter_types.py#L44 Honestly, I bet this had to do with how the database used to be set up. So it turns out that utf8 support in MySQL does not support UTF-8 4 byte multibyte characters (only 1-3 bytes). For example if you do a create image call with an image name to glance with a 4 byte multibyte character in the name it will 500. I'd guess we do something similar in places with the Nova API where we have inadequate input validation. If you want 4 byte support you need to use utf8mb4 instead. Oh... fun. :( I don't know if postgresql has any restrictions (I don't think it does) or if db2 does too. But I don't think we can/should make it a complete free for all. It should at most be what most databases support. I think its a big enough change that this late in the cycle we should push it off to Kilo. It's always much easier to loosen input validation than tighten it (or have to have an oops revert on an officially released Nova). Perhaps some tests to verify that everything we allow past the input validation checks we can actually store. So, honestly, that seems like a pendulum swing in an odd way. Havana use anything you want! Icehouse ? Juno strict asci! Kilo utf8 Correct validation history is Essex: use anything you want! Folsom: strict asci! [..] Juno: strict asci! So I don't think we should make the input validation loose right now to avoid a pendulum swing. Ok, great. That history makes me ok with status quo. I didn't realize it went back so far. Can't we just catch the db exception correctly in glance and not have it explode? And then allow it. Exploding with a 500 on a bad name seems the wrong thing to do anyway. That would also mean that if the user changed their database to support utf8mb4 (which they might want to do if it was important to them) it would all work. I think some release notes would be fine to explain the current situation and limitations. Basically, lets skate towards the puck here, realizing some corner cases exist, but that we're moving in the direction we want to be, not back tracking. One idea is that: How about using base64 encoding/decoding if non-ascii chars come? REST API layer would encode resource names if necessary before passing it to DB layer, and we don't need to consider backend DB features. The disadvantage
Re: [openstack-dev] [nova] are we going to remove the novaclient v3 shell or what?
On Wed, 17 Sep 2014 17:42:30 + Day, Phil philip@hp.com wrote: I think in the hopefully not too distant future we'll be able to make the v1_1 client handle both V2 and V2.1 (who knows. Maybe we can even rename it v2) - and that's what we should do because it will prove if we have full compatibility or not. We should definitely do that cleanup in kilo. Though with microversions hard coding a version number into the directory structure might not make sense. Right now the two things holding that back are the V3 - V2.1 API migration hasn't yet got to the point where the URLs include the tenant ID, and the URL still includes the v3 tag - so the v3 client it the only easy way to exercise the v2.1 API in devstack. For example I needed to do this to test the server group quota changes worked in devstack via V2.1 (I don't trust just getting the unit tests to pass).To do that I had to get boot working and accepting hints, and add support for the limits API. So in the short term it seemed worth pushing those kinds of things back in for now in case they are useful to others. Its not a big overhead to fix the v3 API at the same time (its mostly copied or sub-classed code) until we can add v2.1 support via the v2 client. So the only reason I can think for keeping the v2.1/v3 version of novaclient is if we want to break the python novaclient backwards compatibility to do some cleanups. AFAIK we don't have a policy around python novaclient backwards compatibility and if its ever ok to break it. Chris Phil -Original Message- From: Matt Riedemann [mailto:mrie...@linux.vnet.ibm.com] Sent: 17 September 2014 15:59 To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [nova] are we going to remove the novaclient v3 shell or what? This has come up a couple of times in IRC now but the people that probably know the answer aren't available. There are python-novaclient patches that are adding new CLIs to the v2 (v1_1) and v3 shells, but now that we have the v2.1 API (v2 on v3) why do we still have a v3 shell in the client? Are there plans to remove that? I don't really care either way, but need to know for code reviews. One example: [1] [1] https://review.openstack.org/#/c/108942/ -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Nova API meeting
Hi, Just a reminder that the weekly Nova API meeting is being held tomorrow Friday UTC . We encourage cloud operators and those who use the REST API such as SDK developers and others who and are interested in the future of the API to participate. In other timezones the meeting is at: EST 20:00 (Thu) Japan 09:00 (Fri) China 08:00 (Fri) ACDT 9:30 (Fri) The proposed agenda and meeting details are here: https://wiki.openstack.org/wiki/Meetings/NovaAPI Please feel free to add items to the agenda. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers
On Thu, 4 Sep 2014 11:24:29 +0100 Daniel P. Berrange berra...@redhat.com wrote: - A fairly significant amount of nova code would need to be considered semi-stable API. Certainly everything under nova/virt and any object which is passed in/out of the virt driver API. Changes to such APIs would have to be done in a backwards compatible manner, since it is no longer possible to lock-step change all the virt driver impls. In some ways I think this would be a good thing as it will encourage people to put more thought into the long term maintainability of nova internal code instead of relying on being able to rip it apart later, at will. - The nova/virt/driver.py class would need to be much better specified. All parameters / return values which are opaque dicts must be replaced with objects + attributes. Completion of the objectification work is mandatory, so there is cleaner separation between virt driver impls the rest of Nova. I think for this to work well with multiple repositories and drivers having different priorities over implementing changes in the API it would not just need to be semi-stable, but stable with versioning built in from the start to allow for backwards incompatible changes. And the interface would have to be very well documented including things such as what exceptions are allowed to be raised through the API. Hopefully this would be enforced through code as well. But as long as driver maintainers are willing to commit to this extra overhead I can see it working. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers
On Thu, 4 Sep 2014 12:57:57 -0700 Joe Gordon joe.gord...@gmail.com wrote: Overall I do think we need to re-think how the review burden is distributed. That being said, this is a nice proposal but I am not sure if it moves the review burden around enough or is the right approach. Do you have any rough numbers on what percent of the review burden goes to virt drivers today (how ever you want to define that statement, number of merged patches, man hours, lines of code, number of reviews etc.). If for example today the nova review team spends 10% of there review time on virt drivers then I don't think this proposal will have a significant impact on the review backlog (for nova-common). Even if it doesn't have a huge impact on the review backlog for nova-common (I think it should at least help a bit) it does have the potential to make life much easier for the virt driver developers. I think my main concern is around testing - as soon as we have multiple repositories involved I think debugging of test failures (especially races) tends to get more complicated and we have fewer people who are familiar enough with the two code bases. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] FFE server-group-quotas
I'm willing to sponsor this Chris — Sent from Mailbox On Fri, Sep 5, 2014 at 10:29 PM, Day, Phil philip@hp.com wrote: Hi, I'd like to ask for a FFE for the 3 patchsets that implement quotas for server groups. Server groups (which landed in Icehouse) provides a really useful anti-affinity filter for scheduling that a lot of customers woudl like to use, but without some form of quota control to limit the amount of anti-affinity its impossible to enable it as a feature in a public cloud. The code itself is pretty simple - the number of files touched is a side-effect of having three V2 APIs that report quota information and the need to protect the change in V2 via yet another extension. https://review.openstack.org/#/c/104957/ https://review.openstack.org/#/c/116073/ https://review.openstack.org/#/c/116079/ Phil -Original Message- From: Sahid Orentino Ferdjaoui [mailto:sahid.ferdja...@redhat.com] Sent: 04 September 2014 13:42 To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [nova] FFE request serial-ports Hello, I would like to request a FFE for 4 changesets to complete the blueprint serial-ports. Topic on gerrit: https://review.openstack.org/#/q/status:open+project:openstack/nova+br anch:master+topic:bp/serial-ports,n,z Blueprint on launchpad.net: https://blueprints.launchpad.net/nova/+spec/serial-ports They have already been approved but didn't get enough time to be merged by the gate. Sponsored by: Daniel Berrange Nikola Dipanov s. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] FFE request v2-on-v3-api
Hi, I'd like to request a FFE for 4 changesets from the v2-on-v3-api blueprint: https://review.openstack.org/#/c/113814/ https://review.openstack.org/#/c/115515/ https://review.openstack.org/#/c/115576/ https://review.openstack.org/#/c/11/ They have all already been approved and were in the gate for a while but just didn't quite make it through in time. So they shouldn't put any load on reviewers. Sponsoring cores: Kenichi Ohmichi John Garbutt Me Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [FFE] alternative request for v2-on-v3-api
On Thu, 4 Sep 2014 23:08:09 +0900 Ken'ichi Ohmichi ken1ohmi...@gmail.com wrote: Hi I'd like to request FFE for v2.1 API patches. This request is different from Christopher's one. His request is for the approved patches, but this is for some patches which are not approved yet. https://review.openstack.org/#/c/113169/ : flavor-manage API https://review.openstack.org/#/c/114979/ : quota-sets API https://review.openstack.org/#/c/115197/ : security_groups API I think these API are used in many cases and important, so I'd like to test v2.1 API with them together on RC phase. Two of them have gotten one +2 on each PS and the other one have gotten one +1. I'm happy to sponsor these extra changesets - I've reviewed them all previously. Risk to the rest of Nova is very low. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][FFE] v3-api-schema
On Thu, 4 Sep 2014 22:30:51 +0900 Ken'ichi Ohmichi ken1ohmi...@gmail.com wrote: Hi I'd like to request FFE for patches of v3-api-schema. The list is the following: https://review.openstack.org/#/c/67428/ https://review.openstack.org/#/c/103437/ https://review.openstack.org/#/c/103436/ https://review.openstack.org/#/c/66783/ The one of them has already approved, but it stops merging with temporary -2. The other ones have gotten one +2 on each PS. This work will make v2.1 API strong. I'm happy to sponsor this. This only affects the v2.1 API so is of low risk to the rest of Nova Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Lack of consistency in returning response from tempest clients
On Fri, 29 Aug 2014 11:13:39 -0400
David Kranz dkr...@redhat.com wrote:
On 08/29/2014 10:56 AM, Sean Dague wrote:
On 08/29/2014 10:19 AM, David Kranz wrote:
While reviewing patches for moving response checking to the
clients, I noticed that there are places where client methods do
not return any value. This is usually, but not always, a delete
method. IMO, every rest client method should return at least the
response. Some services return just the response for delete
methods and others return (resp, body). Does any one object to
cleaning this up by just making all client methods return resp,
body? This is mostly a change to the clients. There were only a
few places where a non-delete method was returning just a body
that was used in test code.
Yair and I were discussing this yesterday. As the response
correctness checking is happening deeper in the code (and you are
seeing more and more people assigning the response object to _ ) my
feeling is Tempest clients should probably return a body obj that's
basically.
class ResponseBody(dict):
def __init__(self, body={}, resp=None):
self.update(body)
self.resp = resp
Then all the clients would have single return values, the body
would be the default thing you were accessing (which is usually
what you want), and the response object is accessible if needed to
examine headers.
-Sean
Heh. I agree with that and it is along a similar line to what I
proposed here https://review.openstack.org/#/c/106916/ but using a
dict rather than an attribute dict. I did not propose this since it
is such a big change. All the test code would have to be changed to
remove the resp or _ that is now receiving the response. But I think
we should do this before the client code is moved to tempest-lib.
+1. this would be a nice cleanup.
Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Nova API meeting
Hi, Just a reminder that the weekly Nova API meeting is being held tomorrow Friday UTC . We encourage cloud operators and those who use the REST API such as SDK developers and others who and are interested in the future of the API to participate. In other timezones the meeting is at: EST 20:00 (Thu) Japan 09:00 (Fri) China 08:00 (Fri) ACDT 9:30 (Fri) The proposed agenda and meeting details are here: https://wiki.openstack.org/wiki/Meetings/NovaAPI Please feel free to add items to the agenda. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Group API: add 'action' to authorizer?
On Sat, 23 Aug 2014 03:56:27 -0500 Joe Cropper cropper@gmail.com wrote: Hi Folks, Would anyone be opposed to adding the 'action' checking to the v2/v3 authorizers? This would allow administrators more fine-grained control over who can read vs. create/update/delete server groups. Thoughts? If folks are supportive, I'd be happy to add this... but not sure if we'd treat this as a 'bug' or whether there is a blueprint under which this could be done? Long term we want to have a separate authorizer for every method. Alex had a nova-spec proposed for this but it unfortunately did not make Juno https://review.openstack.org/#/c/92326/ Also since the feature proposal deadline has passed it'll have to wait till Kilo. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA] Picking a Name for the Tempest Library
On Sat, 16 Aug 2014 18:27:19 +0200 Marc Koderer m...@koderer.com wrote: Hi all, Am 15.08.2014 um 23:31 schrieb Jay Pipes jaypi...@gmail.com: I suggest that tempest should be the name of the import'able library, and that the integration tests themselves should be what is pulled out of the current Tempest repository, into their own repo called openstack-integration-tests or os-integration-tests“. why not keeping it simple: tempest: importable test library tempest-tests: all the test cases Simple, obvious and clear ;) +1 Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Nova API meeting
Hi, Just a reminder that the weekly Nova API meeting is being held tomorrow Friday UTC . We encourage cloud operators and those who use the REST API such as SDK developers and others who and are interested in the future of the API to participate. In other timezones the meeting is at: EST 20:00 (Thu) Japan 09:00 (Fri) China 08:00 (Fri) ACDT 9:30 (Fri) The proposed agenda and meeting details are here: https://wiki.openstack.org/wiki/Meetings/NovaAPI Please feel free to add items to the agenda. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On Wed, 13 Aug 2014 18:52:05 -0400 Jay Pipes jaypi...@gmail.com wrote: On 08/13/2014 06:35 PM, Russell Bryant wrote: On 08/13/2014 06:23 PM, Mark McLoughlin wrote: On Wed, 2014-08-13 at 12:05 -0700, James E. Blair wrote: cor...@inaugust.com (James E. Blair) writes: Sean Dague s...@dague.net writes: This has all gone far enough that someone actually wrote a Grease Monkey script to purge all the 3rd Party CI content out of Jenkins UI. People are writing mail filters to dump all the notifications. Dan Berange filters all them out of his gerrit query tools. I should also mention that there is a pending change to do something similar via site-local Javascript in our Gerrit: https://review.openstack.org/#/c/95743/ I don't think it's an ideal long-term solution, but if it works, we may have some immediate relief without all having to install greasemonkey scripts. You may have noticed that this has merged, along with a further change that shows the latest results in a table format. (You may need to force-reload in your browser to see the change.) Beautiful! Thank you so much to everyone involved. +1! Love this. Indeed. Amazeballs. Agreed! This is a really nice improvement Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Group Based Policy and the way forward
On Wed, Aug 6, 2014 at 5:41 PM, Aaron Rosen aaronoro...@gmail.com wrote: On Wed, Aug 6, 2014 at 12:59 AM, Gary Kotton gkot...@vmware.com wrote: From: Aaron Rosen aaronoro...@gmail.com Reply-To: OpenStack List openstack-dev@lists.openstack.org Date: Wednesday, August 6, 2014 at 10:09 AM To: OpenStack List openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Neutron] Group Based Policy and the way forward On Tue, Aug 5, 2014 at 11:18 PM, Gary Kotton gkot...@vmware.com wrote: On 8/5/14, 8:53 PM, Russell Bryant rbry...@redhat.com wrote: On 08/05/2014 01:23 PM, Gary Kotton wrote: Ok, thanks for the clarification. This means that it will not be done automagically as it is today the tenant will need to create a Neutron port and then pass that through. FWIW, that's the direction we've wanted to move in Nova anyway. We'd like to get rid of automatic port creation, but can't do that in the current stable API. Can you elaborate on what you mean here? What are the issues with port creation? Having nova-compute create ports for neutron is problematic if timeouts occur between nova and neutron as you have to garbage collect neutron ports in nova to cleanup (which was the cause of several bug in the cache handing allowing ports to leak into the info_cache in nova). Pushing this out to the tenant is less orchestration nova has to do. [gary] my take on this is that we should allocate this via the n-api and not via the nova compute (which is far too late in the process. But that is another discussion :) I agree, I had actually proposed this here: https://blueprints.launchpad.net/nova/+spec/nova-api-quantum-create-port :), though there are some issues we need to solve in neutron first -- allowing the mac_address on the port to be updated in neutron. This is required for bare metal support as when the port is created we don't know which physical mac will need to be mapped to the port. I think that in the long term (when we can do an API rev) we should just be getting rid of the automatic port creation completely with the updated Nova API. I don't see why the Nova API needs to do proxying work to neutron to create the port when the client can do it directly with neutron (perhaps via some convenience client code if desired). It removes the complexity of the garbage collection on failure issues in Nova that we currently have. Chris -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
