Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Brant Knudson]

On Thu, Oct 2, 2014 at 6:04 AM, Ken'ichi Ohmichi ken1ohmi...@gmail.com
wrote:



 Thanks for your advice, that is very useful input for me.
 I read both keystone-specs and ietf draft-spec for JSON-Home.
 I have a question.

 JSON-Home is useful for advertising API URL paths to clients, I guess
 but it cannot advertise the supported attributes of a request body.
 Is that right?


Right, it says right in the FAQ:
https://tools.ietf.org/html/draft-nottingham-json-home-03#appendix-B.5 :

How Do I find the schema for a format?

   That isn't addressed by home documents. ...


Also, you might want to check out section 5, Representation Hints :
https://tools.ietf.org/html/draft-nottingham-json-home-03#section-5
 . All it says is TBD. So we might have to make up our own standard here.


 For example, we can create a user nobody by passing the following
 request body to Keystone /v2.0/users with POST method:

   '{user: {email: null, password: null, enabled: true, name:
 nobody, tenantId: null}}'

 In this case, I hope Keystone can advertise the above
 attributes(email, name, etc).
 but JSON-Home doesn't cover it as its scope, I guess.


When discussing the document schema I think we're planning to use
JSONSchema... In Keystone, we've got J-S implemented on some parts (I don't
think it covers all resources yet). I also don't think our JSONSchema is
discoverable yet (i.e., you can't download the schema from the server). I
haven't heard of other projects implementing this yet, but maybe someone
has.

There probably is some way to integrate JSON Home with JSONSchema. Maybe
you can put a reference to the JSONSchema in the hints for the resource.

On current Nova v2 API, we need to add dummy extension when adding new
 attributes to the existing request/response body because of
 advertising something
 changed to clients. I'm glad if we can use more standard way for doing it.

 Thanks
 Ken'ichi Ohmichi

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Jay Pipes]

On 10/02/2014 09:54 AM, Brant Knudson wrote:

When discussing the document schema I think we're planning to use
JSONSchema... In Keystone, we've got J-S implemented on some parts (I
don't think it covers all resources yet). I also don't think our
JSONSchema is discoverable yet (i.e., you can't download the schema from
the server). I haven't heard of other projects implementing this yet,
but maybe someone has.


Glance v2 API has had entirely discoverable API via JSONSchema for years 
now:


http://developer.openstack.org/api-ref-image-v2.html#image-schemas-v2

The oscomputevnext API I proposed also has fully discoverable resources 
(JSONSchema+JSON-HAL documents):


http://docs.oscomputevnext.apiary.io/#schema

and paths (JSONHome):

http://docs.oscomputevnext.apiary.io/#openstackcomputeapiroot

So, it's definitely doable and definitely doable without the mess of API 
extensions that litters Nova, Neutron (and Keystone...)


Best,
-jay


There probably is some way to integrate JSON Home with JSONSchema. Maybe
you can put a reference to the JSONSchema in the hints for the resource.

On current Nova v2 API, we need to add dummy extension when adding new
attributes to the existing request/response body because of
advertising something
changed to clients. I'm glad if we can use more standard way for
doing it.

Thanks
Ken'ichi Ohmichi

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
mailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Kenichi Oomichi]

 -Original Message-
 From: Brant Knudson [mailto:b...@acm.org]
 Sent: Thursday, October 02, 2014 10:55 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [Nova] Some ideas for micro-version 
 implementation
 
 Thanks for your advice, that is very useful input for me.
 I read both keystone-specs and ietf draft-spec for JSON-Home.
 I have a question.

 JSON-Home is useful for advertising API URL paths to clients, I guess
 but it cannot advertise the supported attributes of a request body.
 Is that right?
 
 Right, it says right in the FAQ: 
 https://tools.ietf.org/html/draft-nottingham-json-home-03#appendix-B.5 :
 
 How Do I find the schema for a format?

That isn't addressed by home documents. ...
 
 Also, you might want to check out section 5, Representation Hints :
 https://tools.ietf.org/html/draft-nottingham-json-home-03#section-5
 
  . All it says is TBD. So we might have to make up our own standard here.

Thanks again, I got it.
Our own standard seems necessary for this use case.

 For example, we can create a user nobody by passing the following
 request body to Keystone /v2.0/users with POST method:

   '{user: {email: null, password: null, enabled: true, name: 
 nobody, tenantId: null}}'

 In this case, I hope Keystone can advertise the above
 attributes(email, name, etc).
 but JSON-Home doesn't cover it as its scope, I guess.
 
 When discussing the document schema I think we're planning to use 
 JSONSchema... In Keystone, we've got J-S implemented
 on some parts (I don't think it covers all resources yet). I also don't think 
 our JSONSchema is discoverable yet (i.e.,
 you can't download the schema from the server). I haven't heard of other 
 projects implementing this yet, but maybe someone
 has.
 
 There probably is some way to integrate JSON Home with JSONSchema. Maybe you 
 can put a reference to the JSONSchema in
 the hints for the resource.

Oh, the hints is a nice idea for JSONSchema.
Do you have any plan/bp/spec for doing it on keystone?
I'd like to join into it if there is.

Thanks
Ken'ichi Ohmichi

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Kenichi Oomichi]

 -Original Message-
 From: Jay Pipes [mailto:jaypi...@gmail.com]
 Sent: Wednesday, September 24, 2014 12:47 AM
 To: openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [Nova] Some ideas for micro-version 
 implementation
 
 On 09/22/2014 04:27 PM, Brant Knudson wrote:
  On Fri, Sep 19, 2014 at 1:39 AM, Alex Xu x...@linux.vnet.ibm.com
  mailto:x...@linux.vnet.ibm.com wrote:
 
  Close to Kilo, it is time to think about what's next for nova API.
  In Kilo, we
  will continue develop the important feature micro-version.
 
  In previous v2 on v3 propose, it's include some implementations can be
  used for micro-version.
  
  (https://review.openstack.org/__#/c/84695/19/specs/juno/v2-on-__v3-api.rst
  https://review.openstack.org/#/c/84695/19/specs/juno/v2-on-v3-api.rst)
  But finally, those implementations was considered too complex.
 
  So I'm try to find out more simple implementation and solution for
  micro-version.
 
  I wrote down some ideas as blog post at:
  http://soulxu.github.io/blog/__2014/09/12/one-option-for-__nova-api/
  http://soulxu.github.io/blog/2014/09/12/one-option-for-nova-api/
 
  And for those ideas also already done some POC, you can find out in
  the blog post.
 
  As discussion in the Nova API meeting, we want to bring it up to
  mail-list to
  discussion. Hope we can get more idea and option from all developers.
 
  We will appreciate for any comment and suggestion!
 
  Thanks
  Alex
 
 
 
  Did you consider JSON Home[1] for this? For Juno we've got JSON Home
  support in Keystone for Identity v3 (Zaqar was using it already). We
  weren't planning to use it for microversioning since we weren't planning
  on doing microversioning, but I think JSON Home could be used for this
  purpose.
 
  Using JSON Home, you'd have relationships that include the version, then
  the client can check the JSON Home document to see if the server has
  support for the relationship the client wants to use.
 
  [1] http://tools.ietf.org/html/draft-nottingham-json-home-03
 
 ++ I used JSON-Home extensively in the Compute API blueprint I put
 together a few months ago:
 
 http://docs.oscomputevnext.apiary.io/

vNext seems an interesting idea, I thought the implementation way for Nova
a little. API Route Discoverability is a nice design, but a root / URL
will conflict on current list versions API.
Maybe there would be a workaround.

Thanks
Ken'ichi Ohmichi


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Jay Pipes]

On 09/24/2014 05:26 AM, Kenichi Oomichi wrote:

-Original Message-
From: Jay Pipes [mailto:jaypi...@gmail.com]
Sent: Wednesday, September 24, 2014 12:47 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

On 09/22/2014 04:27 PM, Brant Knudson wrote:

On Fri, Sep 19, 2014 at 1:39 AM, Alex Xu x...@linux.vnet.ibm.com
mailto:x...@linux.vnet.ibm.com wrote:

 Close to Kilo, it is time to think about what's next for nova API.
 In Kilo, we
 will continue develop the important feature micro-version.

 In previous v2 on v3 propose, it's include some implementations can be
 used for micro-version.
 (https://review.openstack.org/__#/c/84695/19/specs/juno/v2-on-__v3-api.rst
 https://review.openstack.org/#/c/84695/19/specs/juno/v2-on-v3-api.rst)
 But finally, those implementations was considered too complex.

 So I'm try to find out more simple implementation and solution for
 micro-version.

 I wrote down some ideas as blog post at:
 http://soulxu.github.io/blog/__2014/09/12/one-option-for-__nova-api/
 http://soulxu.github.io/blog/2014/09/12/one-option-for-nova-api/

 And for those ideas also already done some POC, you can find out in
 the blog post.

 As discussion in the Nova API meeting, we want to bring it up to
 mail-list to
 discussion. Hope we can get more idea and option from all developers.

 We will appreciate for any comment and suggestion!

 Thanks
 Alex



Did you consider JSON Home[1] for this? For Juno we've got JSON Home
support in Keystone for Identity v3 (Zaqar was using it already). We
weren't planning to use it for microversioning since we weren't planning
on doing microversioning, but I think JSON Home could be used for this
purpose.

Using JSON Home, you'd have relationships that include the version, then
the client can check the JSON Home document to see if the server has
support for the relationship the client wants to use.

[1] http://tools.ietf.org/html/draft-nottingham-json-home-03


++ I used JSON-Home extensively in the Compute API blueprint I put
together a few months ago:

http://docs.oscomputevnext.apiary.io/


vNext seems an interesting idea, I thought the implementation way for Nova
a little. API Route Discoverability is a nice design, but a root / URL
will conflict on current list versions API.
Maybe there would be a workaround.


Completely agreed, Ken'ichi. The root URL that returns the JSON-Home 
doc in the vNext API is actually *after* the version in the URI, though...


So, the JSON-Home doc would be returned from:

 http://compute.example.com/vNext/

Of course, replacing vNext with v4 or v42 or whatever the next 
major version of the API would be. The real root would still return the 
versions list as it exists today, with a 302 Multiple Choice.


Best,
jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Brant Knudson]

 vNext seems an interesting idea, I thought the implementation way for Nova
 a little. API Route Discoverability is a nice design, but a root / URL
 will conflict on current list versions API.
 Maybe there would be a workaround.


 Completely agreed, Ken'ichi. The root URL that returns the JSON-Home doc
 in the vNext API is actually *after* the version in the URI, though...

 So, the JSON-Home doc would be returned from:

  http://compute.example.com/vNext/

 Of course, replacing vNext with v4 or v42 or whatever the next
 major version of the API would be. The real root would still return the
 versions list as it exists today, with a 302 Multiple Choice.


JSON Home and your JSON versions document can exist on the same path. The
JSON Home response should be returned when the Accept header is
application/json-home[1], and the JSON document when the Accept header is
application/json. Webob makes it easy to support qvalues[2] for the
accept header.

This is how Keystone works for Juno, if you request `/` with Accept:
application/json-home, you get the JSON Home document with paths like
`v3/auth/tokens`. If you request `/v3` with Accept:
application/json-home, you get the JSON Home document with the paths like
`/auth/tokens`. This way, if your auth endpoint is / or /v3 the client can
use the json-home document. # TODO(blk-u): Implement json-home in
keystoneclient.

[1] https://tools.ietf.org/html/draft-nottingham-json-home-03#section-2
[2] http://tools.ietf.org/html/rfc2616#section-3.9

- Brant
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Kenichi Oomichi]

 -Original Message-
 From: Brant Knudson [mailto:b...@acm.org]
 Sent: Thursday, September 25, 2014 12:04 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [Nova] Some ideas for micro-version 
 implementation
 
 
   vNext seems an interesting idea, I thought the implementation 
 way for Nova
   a little. API Route Discoverability is a nice design, but a 
 root / URL
   will conflict on current list versions API.
   Maybe there would be a workaround.
 
 
 
   Completely agreed, Ken'ichi. The root URL that returns the JSON-Home 
 doc in the vNext API is actually *after*
 the version in the URI, though...
 
   So, the JSON-Home doc would be returned from:
 
http://compute.example.com/vNext/ http://compute.example.com/vNext/
 
   Of course, replacing vNext with v4 or v42 or whatever the next 
 major version of the API would be. The
 real root would still return the versions list as it exists today, with a 302 
 Multiple Choice.
 
 
 
 
 JSON Home and your JSON versions document can exist on the same path. The 
 JSON Home response should be returned when the
 Accept header is application/json-home[1], and the JSON document when the 
 Accept header is application/json. Webob
 makes it easy to support qvalues[2] for the accept header.

Thanks Brant, that is a nice advice!
That seems we can avoid conflict between the existing behavior with a accept 
header.
OK, I will consider about Nova with JSON-Home more.

Thanks
Ken'ichi Ohmichi

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Kenichi Oomichi]

 -Original Message-
 From: Jay Pipes [mailto:jaypi...@gmail.com]
 Sent: Wednesday, September 24, 2014 1:09 AM
 To: openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [Nova] Some ideas for micro-version 
 implementation
 
  (1) Input/Output attribute names
  (1.1) These names should be snake_case.
 eg: imageRef - image_ref, flavorRef - flavor_ref, hostId - host_id
 
 Sure, agreed.
 
  (1.2) These names should contain extension names if they are provided in 
  case of some extension loading.
 eg: security_groups - os-security-groups:security_groups
 config_drive - os-config-drive:config_drive
  (1.3) Extension names should consist of hyphens and low chars.
 eg: OS-EXT-AZ:availability_zone - 
  os-extended-availability-zone:availability_zone
 OS-EXT-STS:task_state - os-extended-status:task_state
  (1.4) Extension names should contain the prefix os- if the extension is 
  not core.
 eg: rxtx_factor - os-flavor-rxtx:rxtx_factor
 os-flavor-access:is_public - flavor-access:is_public (flavor-access 
  extension became core)
 
 Frankly, API extensions are a sore on the lip of OpenStack's smile.
 
 They add needless complexity to the API and I believe a discoverable API
 with its resources micro-versioned with schema objects means that API
 extensions should just go the way of the Dodo.

I agree with removing extension names from attribute names as I said on the
other mail. Extension names seems to make attribute name long and complex.

However, the extension *internal* mechanism of v2.1/v3 API is good for me
because we can separate each extension implementation from core part of Nova,
make the code clean and choose necessary extensions as you like. Maybe this
is off-topic because we are discussing external REST API design.


  (1.5) The length of the first attribute depth should be one.
 eg: create a server API with scheduler hints
   -- v2 API input attribute sample 
  ---
 {
 server: {
 imageRef: e5468cc9-3e91-4449-8c4f-e4203c71e365,
 [..]
 },
 OS-SCH-HNT:scheduler_hints: {
 same_host: 5a3dec46-a6e1-4c4d-93c0-8543f5ffe196
 }
 }
   -- v3 API input attribute sample 
  ---
 {
 server: {
 image_ref: e5468cc9-3e91-4449-8c4f-e4203c71e365,
 [..]
 os-scheduler-hints:scheduler_hints: {
 same_host: 5a3dec46-a6e1-4c4d-93c0-8543f5ffe196
 }
 }
 }
 
 -1. What is the point of the containing server attribute in the outer
 dict? Why have it at all?

 Each resource in the REST API should have the same structure: all
 attributes should be top-level, and a special _links attribute should
 be used for the collection of named hyperlinks using the HAL format [1].
 Simple, consistent, and discoverable. No need for the top-level server
 key in the outer dict. Frankly, that is a vestigial tail from the days
 of XML that we should saw off with a dull blade.
 
 [1] http://stateless.co/hal_specification.html. An alternative might be
 the use of JSON-LD for link structuring. See
 https://www.mnot.net/blog/2011/11/25/linking_in_json for a good
 discussion of this topic.

Interesting suggestion, thanks Jay!
I'm not sure HAL is the best at this time, but all flat attributes are very
simple and easy anyway.

  (2) Resource names
  (2.1) Resource names should consist of hyphens and low chars.
 
 ++

thanks, but this seems against Pecan/WSME restriction.
On Pecan/WSME, I think we can specify snake_case only as resource names and 
attributes names.

 eg: /os-instance_usage_audit_log - /os-instance-usage-audit-log
  (2.2) Resource names should contain the prefix os- if the extension is 
  not core.
 eg: /servers/diagnostics - /servers/os-server-diagnostics
 /os-flavor-access - /flavor-access (flavor-access extension became 
  core)
  (2.3) Action names should be snake_case.
 eg: os-getConsoleOutput - get_console_output
 addTenantAccess - add_tenant_access, removeTenantAccess - 
  remove_tenant_access
 
 No extensions, please.

This extensions you said mean server actions?
Current Nova API router is routing server actions based on both API URL and the 
attributes,
and that seems a little complex for me.

Thanks
Ken'ichi Ohmichi


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Christopher Yeoh]

On Wed, 24 Sep 2014 09:04:51 -0400
Jay Pipes jaypi...@gmail.com wrote:

 On 09/24/2014 05:26 AM, Kenichi Oomichi wrote:
  -Original Message-
  From: Jay Pipes [mailto:jaypi...@gmail.com]
  Sent: Wednesday, September 24, 2014 12:47 AM
  To: openstack-dev@lists.openstack.org
  Subject: Re: [openstack-dev] [Nova] Some ideas for micro-version
  implementation
 
  On 09/22/2014 04:27 PM, Brant Knudson wrote:
  On Fri, Sep 19, 2014 at 1:39 AM, Alex Xu x...@linux.vnet.ibm.com
  mailto:x...@linux.vnet.ibm.com wrote:
 
   Close to Kilo, it is time to think about what's next for
  nova API. In Kilo, we
   will continue develop the important feature micro-version.
 
   In previous v2 on v3 propose, it's include some
  implementations can be used for micro-version.
   
  (https://review.openstack.org/__#/c/84695/19/specs/juno/v2-on-__v3-api.rst
   
  https://review.openstack.org/#/c/84695/19/specs/juno/v2-on-v3-api.rst)
   But finally, those implementations was considered too
  complex.
 
   So I'm try to find out more simple implementation and
  solution for micro-version.
 
   I wrote down some ideas as blog post at:
   http://soulxu.github.io/blog/__2014/09/12/one-option-for-__nova-api/
   http://soulxu.github.io/blog/2014/09/12/one-option-for-nova-api/
 
   And for those ideas also already done some POC, you can find
  out in the blog post.
 
   As discussion in the Nova API meeting, we want to bring it
  up to mail-list to
   discussion. Hope we can get more idea and option from all
  developers.
 
   We will appreciate for any comment and suggestion!
 
   Thanks
   Alex
 
 
 
  Did you consider JSON Home[1] for this? For Juno we've got JSON
  Home support in Keystone for Identity v3 (Zaqar was using it
  already). We weren't planning to use it for microversioning since
  we weren't planning on doing microversioning, but I think JSON
  Home could be used for this purpose.
 
  Using JSON Home, you'd have relationships that include the
  version, then the client can check the JSON Home document to see
  if the server has support for the relationship the client wants
  to use.
 
  [1] http://tools.ietf.org/html/draft-nottingham-json-home-03
 
  ++ I used JSON-Home extensively in the Compute API blueprint I put
  together a few months ago:
 
  http://docs.oscomputevnext.apiary.io/
 
  vNext seems an interesting idea, I thought the implementation way
  for Nova a little. API Route Discoverability is a nice design,
  but a root / URL will conflict on current list versions API.
  Maybe there would be a workaround.
 
 Completely agreed, Ken'ichi. The root URL that returns the
 JSON-Home doc in the vNext API is actually *after* the version in the
 URI, though...
 
 So, the JSON-Home doc would be returned from:
 
   http://compute.example.com/vNext/
 
 Of course, replacing vNext with v4 or v42 or whatever the
 next major version of the API would be. The real root would still
 return the versions list as it exists today, with a 302 Multiple
 Choice.

Why have a version in the url at all? With microversions it doesn't
really make any sense and would just cause more churn for app
developers.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Christopher Yeoh]

On Mon, 22 Sep 2014 09:29:26 +
Kenichi Oomichi oomi...@mxs.nes.nec.co.jp wrote:
 
 Before discussing how to implement, I'd like to consider what we
 should implement. IIUC, the purpose of v3 API is to make consistent
 API with the backwards incompatible changes. Through huge discussion
 in Juno cycle, we knew that backwards incompatible changes of REST
 API would be huge pain against clients and we should avoid such
 changes as possible. If new APIs which are consistent in Nova API
 only are inconsistent for whole OpenStack projects, maybe we need to
 change them again for whole OpenStack consistency.

So I think there's three different aspects to microversions which we
can consider quit separately:

- The format of the client header and what the version number means.
  Eg is the version number of the format X.Y.Z, what do we increment
  when we make a bug fix, what do we increment when we make a backwards
  compatible change and what do we increment when we make backwards
  incompatible change.

  Also how does a client request experimental APIs (I believe we have
  consensus that we really need this to avoid backwards incompatible
  changes as much as possible as it allows more testing before
  guaranteeing backwards compatibility)

  I believe that we can consider this part separately from the next two
  issues.

- The implementation on the nova api side. Eg how do we cleanly handle
  supporting multiple versions of the api based on the client header
  (or lack of it which will indicate v2 compatibility. I'll respond
  directly on Alex's original post

- What we are going to use the microversions API feature to do. I think
  they fall under a few broad categories: 

  - Backwards compatible changes. We desperately need a mechanism that
allows us to make backwards compatible changes (eg add another
parameter to a response) without having to add another dummy
extension.

  - Significant backwards incompatible changes. The Tasks API and server
diagnostics API are probably the best examples of this. 

  - V3 like backwards incompatible changes (consistency fixes).

I think getting consensus over backwards compatible changes will be
straightforward. However given the previous v2/v3 discussions I don't
think we will be able to get consensus over doing all or most of the
consistency type fixes even using microversions in the short term.
Because with microversions you get all the changes applied before the
version that you choose. So from a client application point of view its
just as much work as V2 to V3 API transition.

I don't think that means we need to put all of these consistency
changes off forever though. We need to make backwards incompatible
changes in order to implement the Tasks API  and new server
diagnostics api the way we want to. The Tasks API will eventually cover
quite a few interfaces and while say breaking backwards compatibility
with the create server api, we can also fix consistency issues in that
api at the same time. Clients will need to make changes to their app
anyway if they want to take advantage of the new features (or they can
just continue to use the old non-tasks enabled API).

So as we slowly make backwards incompatible changes to the API for
other reasons we can also fix up other issues. Other consistency fixes
we can propose on a case by case basis and the user community can have
input as to whether the cost (app rework) is worth it without getting a
new feature at the same time.

But I think its clear that we *need* the microversions mechanism. So we
don't need to decide beforehand exactly what we're going to use it for
first. I think think its more important that we get a nova-spec
approved for the the first two parts - what it looks like from the
client point of view. And how we're going to implement it.

Regards,

Chris

 
 For avoiding such situation, I think we need to define what is
 consistent REST API across projects. According to Alex's blog, The
 topics might be
 
  - Input/Output attribute names
  - Resource names
  - Status code
 
 The following are hints for making consistent APIs from Nova v3 API
 experience, I'd like to know whether they are the best for API
 consistency.
 
 (1) Input/Output attribute names
 (1.1) These names should be snake_case. 
   eg: imageRef - image_ref, flavorRef - flavor_ref, hostId -
 host_id (1.2) These names should contain extension names if they are
 provided in case of some extension loading. eg: security_groups -
 os-security-groups:security_groups config_drive -
 os-config-drive:config_drive (1.3) Extension names should consist of
 hyphens and low chars. eg: OS-EXT-AZ:availability_zone -
 os-extended-availability-zone:availability_zone OS-EXT-STS:task_state
 - os-extended-status:task_state (1.4) Extension names should contain
 the prefix os- if the extension is not core. eg: rxtx_factor -
 os-flavor-rxtx:rxtx_factor os-flavor-access:is_public -
 flavor-access:is_public (flavor-access extension became core) 

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Christopher Yeoh]

On Mon, 22 Sep 2014 09:47:50 -0500
Anne Gentle a...@openstack.org wrote:
 
  (1) Input/Output attribute names
  (1.1) These names should be snake_case.
eg: imageRef - image_ref, flavorRef - flavor_ref, hostId -
  host_id (1.2) These names should contain extension names if they
  are provided in case of some extension loading.
eg: security_groups - os-security-groups:security_groups
config_drive - os-config-drive:config_drive
 
 
 Do you mean that the os- prefix should be dropped? Or that it should
 be maintained and added as needed?


Originally (I think) the os- prefix was added to avoid potential
name clashes between extensions. Presumably out of tree extensions
too as in-tree ones would be picked up pretty quickly. For a while now
I've been thinking that perhaps we should just drop the os- prefix.
We're trying to discourage optional extensions anyway and I suspect that
the pain of maintaining consistency with os- prefixes is not worth the
benefit.

Where if someone wants to maintain an out of tree plugin they
can bear the burden of avoiding name clashes. This would make it much
easier for both us (nearly no code changes required) and users (no app
changes) when we move some functionality into (or out of) core.

  (1.3) Extension names should consist of hyphens and low chars.
eg: OS-EXT-AZ:availability_zone -
  os-extended-availability-zone:availability_zone
OS-EXT-STS:task_state - os-extended-status:task_state
 
 
 Yes, I don't like the shoutyness of the ALL CAPS.

+1! No to ALL CAPS and contractions or disemvowling of words in order
to save a few bytes.

  (1.4) Extension names should contain the prefix os- if the
  extension is not core.
eg: rxtx_factor - os-flavor-rxtx:rxtx_factor
os-flavor-access:is_public - flavor-access:is_public
  (flavor-access extension became core)
 
 
 Do we have a list of core yet?

We do sort of have a candidate listed hard coded (its a pretty small
list):

API_V3_CORE_EXTENSIONS = set(['consoles',
  'extensions',
  'flavor-access',
  'flavor-extra-specs',
  'flavor-manage',
  'flavors',
  'ips',
  'os-keypairs',
  'server-metadata',
  'servers',
  'versions'])

 
  (3) Status code
  (3.1) Return 201(Created) if a resource creation/updating finishes
  before returning a response.
eg: create a keypair API: 200 - 201
create an agent API: 200 - 201
create an aggregate API: 200 - 201
 
 
 Do you mean a 200 becomes a 201? That's a huge doc impact and SDK
 impact, is it worthwhile? If we do this change, the sooner the
 better, right?

Ideally I think we'd want to do it when breaking a specific part of the
API anyway (for say some new feature). Otherwise its something I think
we should bring up on a case by case with users and operators. Trade
off between returning misleading status codes versus requiring
application side changes (potentially, some may just look for 2xx
anyway).

 
 
 The TC had an action item a while back (a few months) to start an API
 style guide. This seems like a good start. Once the questions are
 discussed I'll get a draft going on the wiki.

So back during the Juno design summit at the cross project API
consistency session we talked about putting together a wiki page with
guidelines for all OpenStack projects. But I think everyone got busy so
not much at all happened. I've had a bit of time recently and tried to
pull together info from the session as well as some other sources and
the start of it is here:

https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines

So perhaps we could merge what is above into this wiki?
It is however still rather Nova specific and we need input from
other projects (I'll send out another email specifically about this). 

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Christopher Yeoh]

On Mon, 22 Sep 2014 15:27:47 -0500
Brant Knudson b...@acm.org wrote:

 
 
 Did you consider JSON Home[1] for this? For Juno we've got JSON Home
 support in Keystone for Identity v3 (Zaqar was using it already). We
 weren't planning to use it for microversioning since we weren't
 planning on doing microversioning, but I think JSON Home could be
 used for this purpose.
 
 Using JSON Home, you'd have relationships that include the version,
 then the client can check the JSON Home document to see if the server
 has support for the relationship the client wants to use.
 

JSON Home is on our list of things to do for the API (but
the priority is lower than microversions). We still need
microversions anyway because we want to be able to support multiple
versions of API through the same resources (ie no url changes). To
make life as easy as possible for users so they are not forced to
upgrade their app whenever we make backwards incompatible changes. I
don't see us supporting every version forever, but it gives them some
time to upgrade.

Regards,

Chris
 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Alex Xu]

On 2014年09月23日 17:12, Christopher Yeoh wrote:

On Mon, 22 Sep 2014 09:29:26 +
Kenichi Oomichi oomi...@mxs.nes.nec.co.jp wrote:

Before discussing how to implement, I'd like to consider what we
should implement. IIUC, the purpose of v3 API is to make consistent
API with the backwards incompatible changes. Through huge discussion
in Juno cycle, we knew that backwards incompatible changes of REST
API would be huge pain against clients and we should avoid such
changes as possible. If new APIs which are consistent in Nova API
only are inconsistent for whole OpenStack projects, maybe we need to
change them again for whole OpenStack consistency.

So I think there's three different aspects to microversions which we
can consider quit separately:

- The format of the client header and what the version number means.
   Eg is the version number of the format X.Y.Z, what do we increment
   when we make a bug fix, what do we increment when we make a backwards
   compatible change and what do we increment when we make backwards
   incompatible change.

   Also how does a client request experimental APIs (I believe we have
   consensus that we really need this to avoid backwards incompatible
   changes as much as possible as it allows more testing before
   guaranteeing backwards compatibility)

   I believe that we can consider this part separately from the next two
   issues.

- The implementation on the nova api side. Eg how do we cleanly handle
   supporting multiple versions of the api based on the client header
   (or lack of it which will indicate v2 compatibility. I'll respond
   directly on Alex's original post

- What we are going to use the microversions API feature to do. I think
   they fall under a few broad categories:

   - Backwards compatible changes. We desperately need a mechanism that
 allows us to make backwards compatible changes (eg add another
 parameter to a response) without having to add another dummy
 extension.

   - Significant backwards incompatible changes. The Tasks API and server
 diagnostics API are probably the best examples of this.

   - V3 like backwards incompatible changes (consistency fixes).

I think getting consensus over backwards compatible changes will be
straightforward. However given the previous v2/v3 discussions I don't
think we will be able to get consensus over doing all or most of the
consistency type fixes even using microversions in the short term.
Because with microversions you get all the changes applied before the
version that you choose. So from a client application point of view its
just as much work as V2 to V3 API transition.

I don't think that means we need to put all of these consistency
changes off forever though. We need to make backwards incompatible
changes in order to implement the Tasks API  and new server
diagnostics api the way we want to. The Tasks API will eventually cover
quite a few interfaces and while say breaking backwards compatibility
with the create server api, we can also fix consistency issues in that
api at the same time. Clients will need to make changes to their app
anyway if they want to take advantage of the new features (or they can
just continue to use the old non-tasks enabled API).

So as we slowly make backwards incompatible changes to the API for
other reasons we can also fix up other issues. Other consistency fixes
we can propose on a case by case basis and the user community can have
input as to whether the cost (app rework) is worth it without getting a
new feature at the same time.


Agree, consistency fixes should depend on the whether the cost is worth 
it or not, maybe we can't fix some inconsistency issues.
And definitely need micro-version for add tasks API and new server 
diagnostics. Also need micro-version to fix some bugs,
like https://bugs.launchpad.net/nova/+bug/1320754 and 
https://bugs.launchpad.net/nova/+bug/1333494.




But I think its clear that we *need* the microversions mechanism. So we
don't need to decide beforehand exactly what we're going to use it for
first. I think think its more important that we get a nova-spec
approved for the the first two parts - what it looks like from the
client point of view. And how we're going to implement it.

Regards,

Chris


For avoiding such situation, I think we need to define what is
consistent REST API across projects. According to Alex's blog, The
topics might be

  - Input/Output attribute names
  - Resource names
  - Status code

The following are hints for making consistent APIs from Nova v3 API
experience, I'd like to know whether they are the best for API
consistency.

(1) Input/Output attribute names
(1.1) These names should be snake_case.
   eg: imageRef - image_ref, flavorRef - flavor_ref, hostId -
host_id (1.2) These names should contain extension names if they are
provided in case of some extension loading. eg: security_groups -
os-security-groups:security_groups config_drive -
os-config-drive:config_drive (1.3) Extension names 

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Jay Pipes]

On 09/22/2014 04:27 PM, Brant Knudson wrote:

On Fri, Sep 19, 2014 at 1:39 AM, Alex Xu x...@linux.vnet.ibm.com
mailto:x...@linux.vnet.ibm.com wrote:

Close to Kilo, it is time to think about what's next for nova API.
In Kilo, we
will continue develop the important feature micro-version.

In previous v2 on v3 propose, it's include some implementations can be
used for micro-version.
(https://review.openstack.org/__#/c/84695/19/specs/juno/v2-on-__v3-api.rst
https://review.openstack.org/#/c/84695/19/specs/juno/v2-on-v3-api.rst)
But finally, those implementations was considered too complex.

So I'm try to find out more simple implementation and solution for
micro-version.

I wrote down some ideas as blog post at:
http://soulxu.github.io/blog/__2014/09/12/one-option-for-__nova-api/
http://soulxu.github.io/blog/2014/09/12/one-option-for-nova-api/

And for those ideas also already done some POC, you can find out in
the blog post.

As discussion in the Nova API meeting, we want to bring it up to
mail-list to
discussion. Hope we can get more idea and option from all developers.

We will appreciate for any comment and suggestion!

Thanks
Alex



Did you consider JSON Home[1] for this? For Juno we've got JSON Home
support in Keystone for Identity v3 (Zaqar was using it already). We
weren't planning to use it for microversioning since we weren't planning
on doing microversioning, but I think JSON Home could be used for this
purpose.

Using JSON Home, you'd have relationships that include the version, then
the client can check the JSON Home document to see if the server has
support for the relationship the client wants to use.

[1] http://tools.ietf.org/html/draft-nottingham-json-home-03


++ I used JSON-Home extensively in the Compute API blueprint I put 
together a few months ago:


http://docs.oscomputevnext.apiary.io/

-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Jay Pipes]

On 09/22/2014 05:29 AM, Kenichi Oomichi wrote:

-Original Message-
From: Alex Xu [mailto:x...@linux.vnet.ibm.com]
Sent: Friday, September 19, 2014 3:40 PM
To: OpenStack Development Mailing List
Subject: [openstack-dev] [Nova] Some ideas for micro-version implementation

Close to Kilo, it is time to think about what's next for nova API. In
Kilo, we
will continue develop the important feature micro-version.

In previous v2 on v3 propose, it's include some implementations can be
used for micro-version.
(https://review.openstack.org/#/c/84695/19/specs/juno/v2-on-v3-api.rst)
But finally, those implementations was considered too complex.

So I'm try to find out more simple implementation and solution for
micro-version.

I wrote down some ideas as blog post at:
http://soulxu.github.io/blog/2014/09/12/one-option-for-nova-api/

And for those ideas also already done some POC, you can find out in the
blog post.

As discussion in the Nova API meeting, we want to bring it up to
mail-list to
discussion. Hope we can get more idea and option from all developers.

We will appreciate for any comment and suggestion!


Before discussing how to implement, I'd like to consider what we should
implement. IIUC, the purpose of v3 API is to make consistent API with the
backwards incompatible changes. Through huge discussion in Juno cycle, we
knew that backwards incompatible changes of REST API would be huge pain
against clients and we should avoid such changes as possible.


Frankly, I believe the lack of perceived value in the v3 API was the 
reason backwards incompatibility was perceived as such a huge pain. v3 
wasn't adding any functionality in the API that wasn't in v2, and v3 was 
bringing along with it much of the crap from the v2 API ... for example, 
API extensions and the ludicrous and needless complexity they add to the 
API.


If the v3 API had given users real additions in functionality, ease of 
use, and clarity, I think users would have embraced a new direction more 
-- especially if support for the v2 API was kept alongside the newer API 
for some period of time. But the v3 API didn't offer anything that 
application and tooling developers cared about. It offered some cleanups 
in return codes, some cleanups in resource naming and parameters, and 
not much else.


 If new APIs

which are consistent in Nova API only are inconsistent for whole OpenStack
projects, maybe we need to change them again for whole OpenStack consistency.

For avoiding such situation, I think we need to define what is consistent
REST API across projects. According to Alex's blog, The topics might be

  - Input/Output attribute names
  - Resource names
  - Status code

The following are hints for making consistent APIs from Nova v3 API experience,
I'd like to know whether they are the best for API consistency.

(1) Input/Output attribute names
(1.1) These names should be snake_case.
   eg: imageRef - image_ref, flavorRef - flavor_ref, hostId - host_id


Sure, agreed.


(1.2) These names should contain extension names if they are provided in case 
of some extension loading.
   eg: security_groups - os-security-groups:security_groups
   config_drive - os-config-drive:config_drive
(1.3) Extension names should consist of hyphens and low chars.
   eg: OS-EXT-AZ:availability_zone - 
os-extended-availability-zone:availability_zone
   OS-EXT-STS:task_state - os-extended-status:task_state
(1.4) Extension names should contain the prefix os- if the extension is not 
core.
   eg: rxtx_factor - os-flavor-rxtx:rxtx_factor
   os-flavor-access:is_public - flavor-access:is_public (flavor-access 
extension became core)


Frankly, API extensions are a sore on the lip of OpenStack's smile.

They add needless complexity to the API and I believe a discoverable API 
with its resources micro-versioned with schema objects means that API 
extensions should just go the way of the Dodo.



(1.5) The length of the first attribute depth should be one.
   eg: create a server API with scheduler hints
 -- v2 API input attribute sample ---
   {
   server: {
   imageRef: e5468cc9-3e91-4449-8c4f-e4203c71e365,
   [..]
   },
   OS-SCH-HNT:scheduler_hints: {
   same_host: 5a3dec46-a6e1-4c4d-93c0-8543f5ffe196
   }
   }
 -- v3 API input attribute sample ---
   {
   server: {
   image_ref: e5468cc9-3e91-4449-8c4f-e4203c71e365,
   [..]
   os-scheduler-hints:scheduler_hints: {
   same_host: 5a3dec46-a6e1-4c4d-93c0-8543f5ffe196
   }
   }
   }


-1. What is the point of the containing server attribute in the outer 
dict? Why have it at all?


Each resource in the REST API should have the same structure: all 
attributes should be top-level, and a special _links attribute should 
be used for the collection of named hyperlinks using

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Kevin L. Mitchell]

On Tue, 2014-09-23 at 12:09 -0400, Jay Pipes wrote:
 I'd like to say finally that I think there should be an OpenStack API 
 working group whose job it is to both pull together a set of OpenStack 
 API practices as well as evaluate new REST APIs proposed in the 
 OpenStack ecosystem to provide guidance to new projects or new 
 subprojects wishing to add resources to an existing REST API.

One of the things that's been bothering me about OpenStack for a while
now is the fact that we have all these different APIs on different
endpoints.  What I've been wondering about is if we should create a
unified ReST API service to replace the service from all of the
individual projects.  Then, users can just hit that one service to
handle all their different interactions.
-- 
Kevin L. Mitchell kevin.mitch...@rackspace.com
Rackspace


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Andrew Laski]

On 09/23/2014 01:15 PM, Kevin L. Mitchell wrote:

On Tue, 2014-09-23 at 12:09 -0400, Jay Pipes wrote:

I'd like to say finally that I think there should be an OpenStack API
working group whose job it is to both pull together a set of OpenStack
API practices as well as evaluate new REST APIs proposed in the
OpenStack ecosystem to provide guidance to new projects or new
subprojects wishing to add resources to an existing REST API.

One of the things that's been bothering me about OpenStack for a while
now is the fact that we have all these different APIs on different
endpoints.  What I've been wondering about is if we should create a
unified ReST API service to replace the service from all of the
individual projects.  Then, users can just hit that one service to
handle all their different interactions.


I've been thinking along very similar lines, but I don't think each 
current API needs to be replaced.  I would very much like to see a 
unified API project that could be responsible for managing requests to 
boot an instance with this network and that volume which would make 
requests to Nova/Neutron/Cinder on the users behalf.




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Louis Taylor]

On Tue, Sep 23, 2014 at 01:32:50PM -0400, Andrew Laski wrote:
 I've been thinking along very similar lines, but I don't think each current
 API needs to be replaced.  I would very much like to see a unified API
 project that could be responsible for managing requests to boot an instance
 with this network and that volume which would make requests to
 Nova/Neutron/Cinder on the users behalf.

Isn't this what openstacksdk [0] is?

[0] https://wiki.openstack.org/wiki/SDK-Development/PythonOpenStackSDK


signature.asc
Description: Digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Kevin L. Mitchell]

On Tue, 2014-09-23 at 18:39 +0100, Louis Taylor wrote:
 On Tue, Sep 23, 2014 at 01:32:50PM -0400, Andrew Laski wrote:
  I've been thinking along very similar lines, but I don't think each current
  API needs to be replaced.  I would very much like to see a unified API
  project that could be responsible for managing requests to boot an instance
  with this network and that volume which would make requests to
  Nova/Neutron/Cinder on the users behalf.
 
 Isn't this what openstacksdk [0] is?
 
 [0] https://wiki.openstack.org/wiki/SDK-Development/PythonOpenStackSDK

Well, openstacksdk is a client, and we're talking about a server.  A
server, in this instance, has some advantages over a client, including
making it easier to create that client in the first place :)
-- 
Kevin L. Mitchell kevin.mitch...@rackspace.com
Rackspace


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Sean Dague]

On 09/23/2014 02:10 PM, Kevin L. Mitchell wrote:
 On Tue, 2014-09-23 at 18:39 +0100, Louis Taylor wrote:
 On Tue, Sep 23, 2014 at 01:32:50PM -0400, Andrew Laski wrote:
 I've been thinking along very similar lines, but I don't think each current
 API needs to be replaced.  I would very much like to see a unified API
 project that could be responsible for managing requests to boot an instance
 with this network and that volume which would make requests to
 Nova/Neutron/Cinder on the users behalf.

 Isn't this what openstacksdk [0] is?

 [0] https://wiki.openstack.org/wiki/SDK-Development/PythonOpenStackSDK
 
 Well, openstacksdk is a client, and we're talking about a server.  A
 server, in this instance, has some advantages over a client, including
 making it easier to create that client in the first place :)

So today we have a proxy API in Nova which does some of this. I guess
the question is is it better to do this in Nova in the future or divorce
it from there.

But regardless of approach, I don't think it really impacts whether or
not we need to solve a saner versioning mechanism than we currently
have... which is randomly add new extensions and pretend it's the same API.

-Sean

-- 
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Chris Friesen]

On 09/23/2014 12:19 PM, Sean Dague wrote:

On 09/23/2014 02:10 PM, Kevin L. Mitchell wrote:

On Tue, 2014-09-23 at 18:39 +0100, Louis Taylor wrote:

On Tue, Sep 23, 2014 at 01:32:50PM -0400, Andrew Laski wrote:

I've been thinking along very similar lines, but I don't think each current
API needs to be replaced.  I would very much like to see a unified API
project that could be responsible for managing requests to boot an instance
with this network and that volume which would make requests to
Nova/Neutron/Cinder on the users behalf.


Isn't this what openstacksdk [0] is?

[0] https://wiki.openstack.org/wiki/SDK-Development/PythonOpenStackSDK


Well, openstacksdk is a client, and we're talking about a server.  A
server, in this instance, has some advantages over a client, including
making it easier to create that client in the first place :)


So today we have a proxy API in Nova which does some of this. I guess
the question is is it better to do this in Nova in the future or divorce
it from there.

But regardless of approach, I don't think it really impacts whether or
not we need to solve a saner versioning mechanism than we currently
have... which is randomly add new extensions and pretend it's the same API.


I find the concept of an API version (i.e. a single version that 
applies to the whole API) to be not particularly useful.


Really what I care about is the API used to accomplish a specific task.

Why not do something like what is done for the userspace/kernel syscall 
API? Userspace code tries to use the most recent one it knows about, if 
that comes back as not a valid syscall then it tries the next older 
version.  As long as trying to use unsupported options fails cleanly, 
there is no ambiguity.


Realistically you'd want to have the complexity hidden behind a helper 
library, but this sort of thing would allow us to add new extensions 
without trying to enforce a version on the API as a whole.


Chris


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Dean Troyer]

On Tue, Sep 23, 2014 at 1:19 PM, Sean Dague s...@dague.net wrote:

 So today we have a proxy API in Nova which does some of this. I guess
 the question is is it better to do this in Nova in the future or divorce
 it from there.


A stand-alone, production-quality configurable proxy would be useful to
take over this work.  It would also be useful for some other things I have
in mind...

For a while now I've wanted to find or write an API proxy/simulator to a)
do API mock-like configurations to actually use and test it over the
network; b) proxy requests on to real servers, with the option of
transforming both request and response for further 'live' API change
mockups.

I'm sure this has already been done, suggestions and pointers to existing
projects welcome.  If not, suggestions for a good starting point/platform.

dt

-- 

Dean Troyer
dtro...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Kenichi Oomichi]

 -Original Message-
 From: Christopher Yeoh [mailto:cbky...@gmail.com]
 Sent: Tuesday, September 23, 2014 7:39 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [Nova] Some ideas for micro-version 
 implementation
 
 On Mon, 22 Sep 2014 09:47:50 -0500
 Anne Gentle a...@openstack.org wrote:
  
   (1) Input/Output attribute names
   (1.1) These names should be snake_case.
 eg: imageRef - image_ref, flavorRef - flavor_ref, hostId -
   host_id (1.2) These names should contain extension names if they
   are provided in case of some extension loading.
 eg: security_groups - os-security-groups:security_groups
 config_drive - os-config-drive:config_drive
  
 
  Do you mean that the os- prefix should be dropped? Or that it should
  be maintained and added as needed?
 
 
 Originally (I think) the os- prefix was added to avoid potential
 name clashes between extensions. Presumably out of tree extensions
 too as in-tree ones would be picked up pretty quickly. For a while now
 I've been thinking that perhaps we should just drop the os- prefix.
 We're trying to discourage optional extensions anyway and I suspect that
 the pain of maintaining consistency with os- prefixes is not worth the
 benefit.

+1 for dropping os- prefix.
On v2.1 API, these prefix makes inconsistency.

   (3) Status code
   (3.1) Return 201(Created) if a resource creation/updating finishes
   before returning a response.
 eg: create a keypair API: 200 - 201
 create an agent API: 200 - 201
 create an aggregate API: 200 - 201
  
 
  Do you mean a 200 becomes a 201? That's a huge doc impact and SDK
  impact, is it worthwhile? If we do this change, the sooner the
  better, right?
 
 Ideally I think we'd want to do it when breaking a specific part of the
 API anyway (for say some new feature). Otherwise its something I think
 we should bring up on a case by case with users and operators. Trade
 off between returning misleading status codes versus requiring
 application side changes (potentially, some may just look for 2xx
 anyway).
 
  
  
  The TC had an action item a while back (a few months) to start an API
  style guide. This seems like a good start. Once the questions are
  discussed I'll get a draft going on the wiki.
 
 So back during the Juno design summit at the cross project API
 consistency session we talked about putting together a wiki page with
 guidelines for all OpenStack projects. But I think everyone got busy so
 not much at all happened. I've had a bit of time recently and tried to
 pull together info from the session as well as some other sources and
 the start of it is here:
 
 https://wiki.openstack.org/wiki/Governance/Proposed/APIGuidelines

Thanks for updating the wiki.
The above wiki is nice for our guideline.

 So perhaps we could merge what is above into this wiki?
 It is however still rather Nova specific and we need input from
 other projects (I'll send out another email specifically about this).

I agree, much input is necessary from other projects also for whole
OpenStack consistency.

Thanks
Ken'ichi Ohmichi


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Kenichi Oomichi]

Hi Alex,

Thank you for doing this.

 -Original Message-
 From: Alex Xu [mailto:x...@linux.vnet.ibm.com]
 Sent: Friday, September 19, 2014 3:40 PM
 To: OpenStack Development Mailing List
 Subject: [openstack-dev] [Nova] Some ideas for micro-version implementation
 
 Close to Kilo, it is time to think about what's next for nova API. In
 Kilo, we
 will continue develop the important feature micro-version.
 
 In previous v2 on v3 propose, it's include some implementations can be
 used for micro-version.
 (https://review.openstack.org/#/c/84695/19/specs/juno/v2-on-v3-api.rst)
 But finally, those implementations was considered too complex.
 
 So I'm try to find out more simple implementation and solution for
 micro-version.
 
 I wrote down some ideas as blog post at:
 http://soulxu.github.io/blog/2014/09/12/one-option-for-nova-api/
 
 And for those ideas also already done some POC, you can find out in the
 blog post.
 
 As discussion in the Nova API meeting, we want to bring it up to
 mail-list to
 discussion. Hope we can get more idea and option from all developers.
 
 We will appreciate for any comment and suggestion!

Before discussing how to implement, I'd like to consider what we should
implement. IIUC, the purpose of v3 API is to make consistent API with the
backwards incompatible changes. Through huge discussion in Juno cycle, we
knew that backwards incompatible changes of REST API would be huge pain
against clients and we should avoid such changes as possible. If new APIs
which are consistent in Nova API only are inconsistent for whole OpenStack
projects, maybe we need to change them again for whole OpenStack consistency.

For avoiding such situation, I think we need to define what is consistent
REST API across projects. According to Alex's blog, The topics might be

 - Input/Output attribute names
 - Resource names
 - Status code

The following are hints for making consistent APIs from Nova v3 API experience,
I'd like to know whether they are the best for API consistency.

(1) Input/Output attribute names
(1.1) These names should be snake_case. 
  eg: imageRef - image_ref, flavorRef - flavor_ref, hostId - host_id
(1.2) These names should contain extension names if they are provided in case 
of some extension loading.
  eg: security_groups - os-security-groups:security_groups
  config_drive - os-config-drive:config_drive
(1.3) Extension names should consist of hyphens and low chars.
  eg: OS-EXT-AZ:availability_zone - 
os-extended-availability-zone:availability_zone
  OS-EXT-STS:task_state - os-extended-status:task_state
(1.4) Extension names should contain the prefix os- if the extension is not 
core.
  eg: rxtx_factor - os-flavor-rxtx:rxtx_factor
  os-flavor-access:is_public - flavor-access:is_public (flavor-access 
extension became core)
(1.5) The length of the first attribute depth should be one.
  eg: create a server API with scheduler hints
-- v2 API input attribute sample ---
  {
  server: {
  imageRef: e5468cc9-3e91-4449-8c4f-e4203c71e365,
  [..]
  },
  OS-SCH-HNT:scheduler_hints: {
  same_host: 5a3dec46-a6e1-4c4d-93c0-8543f5ffe196
  }
  }
-- v3 API input attribute sample ---
  {
  server: {
  image_ref: e5468cc9-3e91-4449-8c4f-e4203c71e365,
  [..]
  os-scheduler-hints:scheduler_hints: {
  same_host: 5a3dec46-a6e1-4c4d-93c0-8543f5ffe196
  }
  }
  }

(2) Resource names
(2.1) Resource names should consist of hyphens and low chars.
  eg: /os-instance_usage_audit_log - /os-instance-usage-audit-log
(2.2) Resource names should contain the prefix os- if the extension is not 
core.
  eg: /servers/diagnostics - /servers/os-server-diagnostics
  /os-flavor-access - /flavor-access (flavor-access extension became core)
(2.3) Action names should be snake_case.
  eg: os-getConsoleOutput - get_console_output
  addTenantAccess - add_tenant_access, removeTenantAccess - 
remove_tenant_access

(3) Status code
(3.1) Return 201(Created) if a resource creation/updating finishes before 
returning a response.
  eg: create a keypair API: 200 - 201
  create an agent API: 200 - 201
  create an aggregate API: 200 - 201
(3.2) Return 204(No Content) if a resource deletion finishes before returning a 
response.
  eg: delete a keypair API: 200 - 204
  delete an agent API: 200 - 204
  delete an aggregate API: 200 - 204
(3.3) Return 202(Accepted) if a request doesn't finish yet before returning a 
response.
  eg: rescue a server API: 200 -202

Any comments are welcome.

Thanks
Ken'ichi Ohmichi

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Anne Gentle]

On Mon, Sep 22, 2014 at 4:29 AM, Kenichi Oomichi oomi...@mxs.nes.nec.co.jp
wrote:

 Hi Alex,

 Thank you for doing this.

  -Original Message-
  From: Alex Xu [mailto:x...@linux.vnet.ibm.com]
  Sent: Friday, September 19, 2014 3:40 PM
  To: OpenStack Development Mailing List
  Subject: [openstack-dev] [Nova] Some ideas for micro-version
 implementation
 
  Close to Kilo, it is time to think about what's next for nova API. In
  Kilo, we
  will continue develop the important feature micro-version.
 
  In previous v2 on v3 propose, it's include some implementations can be
  used for micro-version.
  (https://review.openstack.org/#/c/84695/19/specs/juno/v2-on-v3-api.rst)
  But finally, those implementations was considered too complex.
 
  So I'm try to find out more simple implementation and solution for
  micro-version.
 
  I wrote down some ideas as blog post at:
  http://soulxu.github.io/blog/2014/09/12/one-option-for-nova-api/
 
  And for those ideas also already done some POC, you can find out in the
  blog post.
 
  As discussion in the Nova API meeting, we want to bring it up to
  mail-list to
  discussion. Hope we can get more idea and option from all developers.
 
  We will appreciate for any comment and suggestion!


I would greatly appreciate this style guide to be finalized for
documentation purposes as well. Thanks for starting this write-up. I'd be
happy to write it up on a wiki page while we get agreement, would that be
helpful?



 Before discussing how to implement, I'd like to consider what we should
 implement. IIUC, the purpose of v3 API is to make consistent API with the
 backwards incompatible changes. Through huge discussion in Juno cycle, we
 knew that backwards incompatible changes of REST API would be huge pain
 against clients and we should avoid such changes as possible. If new APIs
 which are consistent in Nova API only are inconsistent for whole OpenStack
 projects, maybe we need to change them again for whole OpenStack
 consistency.

 For avoiding such situation, I think we need to define what is consistent
 REST API across projects. According to Alex's blog, The topics might be

  - Input/Output attribute names
  - Resource names
  - Status code

 The following are hints for making consistent APIs from Nova v3 API
 experience,
 I'd like to know whether they are the best for API consistency.

 (1) Input/Output attribute names
 (1.1) These names should be snake_case.
   eg: imageRef - image_ref, flavorRef - flavor_ref, hostId - host_id
 (1.2) These names should contain extension names if they are provided in
 case of some extension loading.
   eg: security_groups - os-security-groups:security_groups
   config_drive - os-config-drive:config_drive


Do you mean that the os- prefix should be dropped? Or that it should be
maintained and added as needed?


 (1.3) Extension names should consist of hyphens and low chars.
   eg: OS-EXT-AZ:availability_zone -
 os-extended-availability-zone:availability_zone
   OS-EXT-STS:task_state - os-extended-status:task_state


Yes, I don't like the shoutyness of the ALL CAPS.


 (1.4) Extension names should contain the prefix os- if the extension is
 not core.
   eg: rxtx_factor - os-flavor-rxtx:rxtx_factor
   os-flavor-access:is_public - flavor-access:is_public (flavor-access
 extension became core)


Do we have a list of core yet?


 (1.5) The length of the first attribute depth should be one.
   eg: create a server API with scheduler hints
 -- v2 API input attribute sample
 ---
   {
   server: {
   imageRef: e5468cc9-3e91-4449-8c4f-e4203c71e365,
   [..]
   },
   OS-SCH-HNT:scheduler_hints: {
   same_host: 5a3dec46-a6e1-4c4d-93c0-8543f5ffe196
   }
   }
 -- v3 API input attribute sample
 ---
   {
   server: {
   image_ref: e5468cc9-3e91-4449-8c4f-e4203c71e365,
   [..]
   os-scheduler-hints:scheduler_hints: {
   same_host: 5a3dec46-a6e1-4c4d-93c0-8543f5ffe196
   }
   }
   }

 (2) Resource names
 (2.1) Resource names should consist of hyphens and low chars.
   eg: /os-instance_usage_audit_log - /os-instance-usage-audit-log
 (2.2) Resource names should contain the prefix os- if the extension is
 not core.
   eg: /servers/diagnostics - /servers/os-server-diagnostics
   /os-flavor-access - /flavor-access (flavor-access extension became
 core)
 (2.3) Action names should be snake_case.
   eg: os-getConsoleOutput - get_console_output
   addTenantAccess - add_tenant_access, removeTenantAccess -
 remove_tenant_access


Yes to all of these.


 (3) Status code
 (3.1) Return 201(Created) if a resource creation/updating finishes before
 returning a response.
   eg: create a keypair API: 200 - 201
   create an agent API: 200 - 201
   create an aggregate API: 200 - 201


Do you mean a 200

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Brant Knudson]

On Fri, Sep 19, 2014 at 1:39 AM, Alex Xu x...@linux.vnet.ibm.com wrote:

 Close to Kilo, it is time to think about what's next for nova API. In
 Kilo, we
 will continue develop the important feature micro-version.

 In previous v2 on v3 propose, it's include some implementations can be
 used for micro-version.
 (https://review.openstack.org/#/c/84695/19/specs/juno/v2-on-v3-api.rst)
 But finally, those implementations was considered too complex.

 So I'm try to find out more simple implementation and solution for
 micro-version.

 I wrote down some ideas as blog post at:
 http://soulxu.github.io/blog/2014/09/12/one-option-for-nova-api/

 And for those ideas also already done some POC, you can find out in the
 blog post.

 As discussion in the Nova API meeting, we want to bring it up to mail-list
 to
 discussion. Hope we can get more idea and option from all developers.

 We will appreciate for any comment and suggestion!

 Thanks
 Alex



Did you consider JSON Home[1] for this? For Juno we've got JSON Home
support in Keystone for Identity v3 (Zaqar was using it already). We
weren't planning to use it for microversioning since we weren't planning on
doing microversioning, but I think JSON Home could be used for this purpose.

Using JSON Home, you'd have relationships that include the version, then
the client can check the JSON Home document to see if the server has
support for the relationship the client wants to use.

[1] http://tools.ietf.org/html/draft-nottingham-json-home-03

- Brant
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Alex Xu]

On 2014年09月23日 04:27, Brant Knudson wrote:



On Fri, Sep 19, 2014 at 1:39 AM, Alex Xu x...@linux.vnet.ibm.com 
mailto:x...@linux.vnet.ibm.com wrote:


Close to Kilo, it is time to think about what's next for nova API.
In Kilo, we
will continue develop the important feature micro-version.

In previous v2 on v3 propose, it's include some implementations can be
used for micro-version.
(https://review.openstack.org/#/c/84695/19/specs/juno/v2-on-v3-api.rst)
But finally, those implementations was considered too complex.

So I'm try to find out more simple implementation and solution for
micro-version.

I wrote down some ideas as blog post at:
http://soulxu.github.io/blog/2014/09/12/one-option-for-nova-api/

And for those ideas also already done some POC, you can find out
in the blog post.

As discussion in the Nova API meeting, we want to bring it up to
mail-list to
discussion. Hope we can get more idea and option from all developers.

We will appreciate for any comment and suggestion!

Thanks
Alex



Did you consider JSON Home[1] for this? For Juno we've got JSON Home 
support in Keystone for Identity v3 (Zaqar was using it already). We 
weren't planning to use it for microversioning since we weren't 
planning on doing microversioning, but I think JSON Home could be used 
for this purpose.


Using JSON Home, you'd have relationships that include the version, 
then the client can check the JSON Home document to see if the server 
has support for the relationship the client wants to use.


[1] http://tools.ietf.org/html/draft-nottingham-json-home-03

- Brant

Brant, thanks for your comment!

In the micro-version spec discussion, there are discussion JSON-Home 
also. (At line 158 
https://review.openstack.org/#/c/101648/1/specs/juno/api-microversions-alt.rst)

And people like it. JSON-Home is good choice for API discoverable.

That didn't describe in the blog post. The blog post is most focus on 
implement multiple version support in the wsgi infrastructure. But the 
proposed implementation is good for support API discoverable, it propose 
defining json-schema for each version API, not use tranlsation stuff. 
Then the json-schema can be exposed to user easily (I guess Json-schema 
in the Json-home), make the API discoverable.


The micro-version specs is also need discussion in the K, it defined how 
can we bump the version, how to interact with client. But whatever the 
wsgi infrastructure need support multiple version. So I begin to think 
about it.






___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some ideas for micro-version implementation

[Eli Qiao]

? 2014?09?22? 22:47, Anne Gentle ??:



On Mon, Sep 22, 2014 at 4:29 AM, Kenichi Oomichi 
oomi...@mxs.nes.nec.co.jp mailto:oomi...@mxs.nes.nec.co.jp wrote:


Hi Alex,

Thank you for doing this.

 -Original Message-
 From: Alex Xu [mailto:x...@linux.vnet.ibm.com
mailto:x...@linux.vnet.ibm.com]
 Sent: Friday, September 19, 2014 3:40 PM
 To: OpenStack Development Mailing List
 Subject: [openstack-dev] [Nova] Some ideas for micro-version
implementation

 Close to Kilo, it is time to think about what's next for nova
API. In
 Kilo, we
 will continue develop the important feature micro-version.

 In previous v2 on v3 propose, it's include some implementations
can be
 used for micro-version.

(https://review.openstack.org/#/c/84695/19/specs/juno/v2-on-v3-api.rst)
 But finally, those implementations was considered too complex.

 So I'm try to find out more simple implementation and solution for
 micro-version.

 I wrote down some ideas as blog post at:
 http://soulxu.github.io/blog/2014/09/12/one-option-for-nova-api/

 And for those ideas also already done some POC, you can find out
in the
 blog post.

 As discussion in the Nova API meeting, we want to bring it up to
 mail-list to
 discussion. Hope we can get more idea and option from all
developers.

 We will appreciate for any comment and suggestion!


I would greatly appreciate this style guide to be finalized for 
documentation purposes as well. Thanks for starting this write-up. I'd 
be happy to write it up on a wiki page while we get agreement, would 
that be helpful?



Before discussing how to implement, I'd like to consider what we
should
implement. IIUC, the purpose of v3 API is to make consistent API
with the
backwards incompatible changes. Through huge discussion in Juno
cycle, we
knew that backwards incompatible changes of REST API would be huge
pain
against clients and we should avoid such changes as possible. If
new APIs
which are consistent in Nova API only are inconsistent for whole
OpenStack
projects, maybe we need to change them again for whole OpenStack
consistency.

For avoiding such situation, I think we need to define what is
consistent
REST API across projects. According to Alex's blog, The topics
might be

 - Input/Output attribute names
 - Resource names
 - Status code

The following are hints for making consistent APIs from Nova v3
API experience,
I'd like to know whether they are the best for API consistency.

(1) Input/Output attribute names
(1.1) These names should be snake_case.
  eg: imageRef - image_ref, flavorRef - flavor_ref, hostId -
host_id
(1.2) These names should contain extension names if they are
provided in case of some extension loading.
  eg: security_groups - os-security-groups:security_groups
  config_drive - os-config-drive:config_drive


Do you mean that the os- prefix should be dropped? Or that it should 
be maintained and added as needed?


(1.3) Extension names should consist of hyphens and low chars.
  eg: OS-EXT-AZ:availability_zone -
os-extended-availability-zone:availability_zone
  OS-EXT-STS:task_state - os-extended-status:task_state


Yes, I don't like the shoutyness of the ALL CAPS.

(1.4) Extension names should contain the prefix os- if the
extension is not core.
  eg: rxtx_factor - os-flavor-rxtx:rxtx_factor
  os-flavor-access:is_public - flavor-access:is_public
(flavor-access extension became core)


Do we have a list of core yet?

(1.5) The length of the first attribute depth should be one.
  eg: create a server API with scheduler hints
-- v2 API input attribute sample
---
  {
  server: {
  imageRef: e5468cc9-3e91-4449-8c4f-e4203c71e365,
  [..]
  },
  OS-SCH-HNT:scheduler_hints: {
  same_host: 5a3dec46-a6e1-4c4d-93c0-8543f5ffe196
  }
  }
-- v3 API input attribute sample
---
  {
  server: {
  image_ref: e5468cc9-3e91-4449-8c4f-e4203c71e365,
  [..]
  os-scheduler-hints:scheduler_hints: {
  same_host: 5a3dec46-a6e1-4c4d-93c0-8543f5ffe196
  }
  }
  }

(2) Resource names
(2.1) Resource names should consist of hyphens and low chars.
  eg: /os-instance_usage_audit_log - /os-instance-usage-audit-log
(2.2) Resource names should contain the prefix os- if the
extension is not core.
  eg: /servers/diagnostics - /servers/os-server-diagnostics
  /os-flavor-access - /flavor-access (flavor-access extension
became core

[openstack-dev] [Nova] Some ideas for micro-version implementation

[Alex Xu]

Close to Kilo, it is time to think about what's next for nova API. In 
Kilo, we

will continue develop the important feature micro-version.

In previous v2 on v3 propose, it's include some implementations can be
used for micro-version.
(https://review.openstack.org/#/c/84695/19/specs/juno/v2-on-v3-api.rst)
But finally, those implementations was considered too complex.

So I'm try to find out more simple implementation and solution for 
micro-version.


I wrote down some ideas as blog post at:
http://soulxu.github.io/blog/2014/09/12/one-option-for-nova-api/

And for those ideas also already done some POC, you can find out in the 
blog post.


As discussion in the Nova API meeting, we want to bring it up to 
mail-list to

discussion. Hope we can get more idea and option from all developers.

We will appreciate for any comment and suggestion!

Thanks
Alex


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev