Re: [openstack-dev] [Nova][Tempest] Tempest will deny extra properties on Nova v2/v2.1 API

2015-02-24 Thread David Kranz

On 02/24/2015 06:55 AM, Ken'ichi Ohmichi wrote:

Hi Ghanshyam,

2015-02-24 20:28 GMT+09:00 GHANSHYAM MANN ghanshyamm...@gmail.com:

On Tue, Feb 24, 2015 at 6:48 PM, Ken'ichi Ohmichi ken1ohmi...@gmail.com
wrote:

Hi

Nova team is developing Nova v2.1 API + microversions in this cycle,
and the status of Nova v2.1 API has been changed to CURRENT from
EXPERIMENTAL.
That said new API properties should be added via microversions, and
v2/v2.1 API(*without* microversions) should return the same response
without any new properties.
Now Tempest allows extra properties of a Nova API response because we
thought Tempest should not block Nova API development.

However, I think Tempest needs to deny extra properties in
non-microversions test cases because we need to block accidental
changes of v2/v2.1 API and encourage to use microversions for API
changes.
https://review.openstack.org/#/c/156130/ is trying to do that, but I'd
like to get opinions before that.

If the above change is merged, we can not use Tempest on OpenStack
environments which provide the original properties.


I think that will be nice to block additional properties.

Do you mean OpenStack environment with micro-versions enabled?
In those cases too tempest should run successfully as it requests on V2 or
V2.1 endpoint not on microversion.

My previous words were unclear, sorry.
The above OpenStack environment means the environment which is
customized by a cloud service provider and it returns a response which
includes the provider original properties.

On microversions discussion, we considered the customized API by
a cloud service provider for the design. Then I guess there are some
environments return extra properties and Tempest will deny them if
the patch is merged. I'd like to know the situation is acceptable or not
as Tempest purpose.
Ken'ichi, can you please provide a pointer to the referenced 
microversions discussion and/or summarize the conclusion?


The commit message is saying that returning extra values without a new 
microversion is an incompatible (disallowed) change. This was already 
true, unless creating a new extension, according to 
https://wiki.openstack.org/wiki/APIChangeGuidelines.


Seems to me that extra properties (unless using a syntax marking them as 
such), are either allowed or not. If not, tempest should fail on them. 
If service providers are allowed to add returned properties, and not 
required to use some special syntax to distinguish them, that is a bad 
api. If tempest can't tell the difference between a legitimate added 
property and some one misspelling while returning an optional property, 
I'm not sure how we test for the unintentional change case.


 -David



Thanks
Ken Ohmichi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][Tempest] Tempest will deny extra properties on Nova v2/v2.1 API

2015-02-24 Thread Kenichi Oomichi
Hi David,

 -Original Message-
 From: David Kranz [mailto:dkr...@redhat.com]
 Sent: Wednesday, February 25, 2015 4:19 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [Nova][Tempest] Tempest will deny extra 
 properties on Nova v2/v2.1 API
 
  2015-02-24 20:28 GMT+09:00 GHANSHYAM MANN ghanshyamm...@gmail.com:
  On Tue, Feb 24, 2015 at 6:48 PM, Ken'ichi Ohmichi ken1ohmi...@gmail.com
  wrote:
  Hi
 
  Nova team is developing Nova v2.1 API + microversions in this cycle,
  and the status of Nova v2.1 API has been changed to CURRENT from
  EXPERIMENTAL.
  That said new API properties should be added via microversions, and
  v2/v2.1 API(*without* microversions) should return the same response
  without any new properties.
  Now Tempest allows extra properties of a Nova API response because we
  thought Tempest should not block Nova API development.
 
  However, I think Tempest needs to deny extra properties in
  non-microversions test cases because we need to block accidental
  changes of v2/v2.1 API and encourage to use microversions for API
  changes.
  https://review.openstack.org/#/c/156130/ is trying to do that, but I'd
  like to get opinions before that.
 
  If the above change is merged, we can not use Tempest on OpenStack
  environments which provide the original properties.
 
  I think that will be nice to block additional properties.
 
  Do you mean OpenStack environment with micro-versions enabled?
  In those cases too tempest should run successfully as it requests on V2 or
  V2.1 endpoint not on microversion.
  My previous words were unclear, sorry.
  The above OpenStack environment means the environment which is
  customized by a cloud service provider and it returns a response which
  includes the provider original properties.
 
  On microversions discussion, we considered the customized API by
  a cloud service provider for the design. Then I guess there are some
  environments return extra properties and Tempest will deny them if
  the patch is merged. I'd like to know the situation is acceptable or not
  as Tempest purpose.

 Ken'ichi, can you please provide a pointer to the referenced
 microversions discussion and/or summarize the conclusion?

OK, Christopher Yeoh mentioned about future microversions on
http://lists.openstack.org/pipermail/openstack-dev/2015-February/057390.html

Now there are two API versions in Nova: v2 and v2.1.
v2 is implemented with old framework and we will remove it in long term.
v2.1 is implemented with new framework and it is the same as v2 as API
behaviors for providing full v2 compatibility.
Microversions will be v2.2, v2.3, ... and we can change API behaviors which
are both backwards compatible and incompatible.
Clients need to specify the preferred microversion in the request header if
they want to use microversioned API.
If not specifying it, Nova works as default API version which is v2.1.
Tempest doesn't specifies it now. So Tempest should check the behavior which
is the same as v2 on default API(v2.1) tests.

As Chris said no changes should be accepted to the old v2 api code on the
above mail, that means backwards compatible changes also should not be applied
to v2 and v2.1.

 The commit message is saying that returning extra values without a new
 microversion is an incompatible (disallowed) change.
 This was already true, unless creating a new extension, according to
 https://wiki.openstack.org/wiki/APIChangeGuidelines.

Yeah, a nice point.
We added a new dummy extension when adding new extra properties for the
above guideline, and extension numbers increased. We can use microversions
instead of adding more dummy extensions.

 Seems to me that extra properties (unless using a syntax marking them as
 such), are either allowed or not. If not, tempest should fail on them.
 If service providers are allowed to add returned properties, and not
 required to use some special syntax to distinguish them, that is a bad
 api. If tempest can't tell the difference between a legitimate added
 property and some one misspelling while returning an optional property,
 I'm not sure how we test for the unintentional change case.

We had discussed *vendor* flag in Kilo summit, and we have decided drop
of the flag[1]. So we don't have a standard way to know the difference.
I feel now Tempest can deny extra properties as upstream development.

[1]: https://etherpad.openstack.org/p/kilo-nova-microversions


Thanks
Ken Ohmichi


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev