Re: [openstack-dev] [nova] [tempest] [ceilometer] Defining Diagnotics API Schema (was: looking for clarification...)
Open Ended API isn't really an API. An API has a contract, otherwise it isn't an API. The diagnostics REST call currently seems to have no contract at all, and is just implemented by the underlying driver to whatever seems like a good idea today (also, no versioning on things, so what libvirt returns in grizzly vs. havana is up in the air). At minimum we need some kind of per driver (probably with a version) json schema so you could at least identify over the wire what you should be expecting. Even better would be a generic definition, but I get that takes time / effort. I think the approach at this point is a bug on nova has been filed - https://bugs.launchpad.net/nova/+bug/1240043 ... and we're going to skip the tempest tests entirely based on that bug. Testing an API, that isn't actually an API, is really beyond tempest scope. On Mon, Oct 14, 2013 at 4:54 AM, Bob Ball bob.b...@citrix.com wrote: I’m happy with that approach – again I’ve not seen any discussions about how this should be done. ** ** I’ve added [tempest] and [ceilometer] tags so we can hopefully get input from the guys involved. ** ** Bob ** ** *From:* Gary Kotton [mailto:gkot...@vmware.com] *Sent:* 13 October 2013 05:21 *To:* OpenStack Development Mailing List *Subject:* Re: [openstack-dev] [nova] Looking for clarification on the diagnostics API ** ** Hi, I agree with Matt here. This is not broad enough. One option is to have a tempest class that overrides for various backend plugins. Then the test can be haredednd for each driver. I am not sure if that is something that has been talked about. Thanks Gary ** ** *From: *Matt Riedemann mrie...@us.ibm.com *Reply-To: *OpenStack Development Mailing List openstack-dev@lists.openstack.org *Date: *Sunday, October 13, 2013 6:13 AM *To: *OpenStack Development Mailing List openstack-dev@lists.openstack.org *Subject: *Re: [openstack-dev] [nova] Looking for clarification on the diagnostics API ** ** There is also a tempest patch now to ease some of the libvirt-specific keys checked in the new diagnostics tests there: https://review.openstack.org/#/c/51412/ To relay some of my concerns that I put in that patch: *I'm not sure how I feel about this. It should probably be more generic but I think we need more than just a change in tempest to enforce it, i.e. we should have a nova patch that changes the doc strings for the abstract compute driver method to specify what the minimum keys are for the info returned, maybe a doc api sample change, etc?* *For reference, here is the mailing list post I started on this last week: * * http://lists.openstack.org/pipermail/openstack-dev/2013-October/016385.html *http://lists.openstack.org/pipermail/openstack-dev/2013-October/016385.html *There are also docs here (these examples use xen and libvirt):* * http://docs.openstack.org/grizzly/openstack-compute/admin/content/configuring-openstack-compute-basics.html *http://docs.openstack.org/grizzly/openstack-compute/admin/content/configuring-openstack-compute-basics.html *And under procedure 4.4 here:* * http://docs.openstack.org/admin-guide-cloud/content/ch_introduction-to-openstack-compute.html#section_manage-the-cloud *http://docs.openstack.org/admin-guide-cloud/content/ch_introduction-to-openstack-compute.html#section_manage-the-cloud = I also found this wiki page related to metering and the nova diagnostics API: https://wiki.openstack.org/wiki/EfficientMetering/FutureNovaInteractionModel So it seems like if at some point this will be used with ceilometer it should be standardized a bit which is what the Tempest part starts but I don't want it to get lost there. Thanks, *MATT RIEDEMANN* Advisory Software Engineer Cloud Solutions and OpenStack Development -- *Phone:* 1-507-253-7622 | *Mobile:* 1-507-990-1889* E-mail:* mrie...@us.ibm.com [image: IBM] 3605 Hwy 52 N Rochester, MN 55901-1407 United States From:Gary Kotton gkot...@vmware.com To:OpenStack Development Mailing List openstack-dev@lists.openstack.org, Date:10/12/2013 01:42 PM Subject:Re: [openstack-dev] [nova] Looking for clarification on the diagnostics API -- Yup, it seems to be hypervisor specific. I have added in the Vmware support following you correcting in the Vmware driver. Thanks Gary *From: *Matt Riedemann mrie...@us.ibm.com* Reply-To: *OpenStack Development Mailing List openstack-dev@lists.openstack.org* Date: *Thursday, October 10, 2013 10:17 PM* To: *OpenStack Development Mailing List openstack-dev@lists.openstack.org * Subject: *Re: [openstack-dev] [nova] Looking for clarification on the diagnostics API Looks like this has been brought up a couple of times: * *https://lists.launchpad.net/openstack/msg09138.html *
Re: [openstack-dev] [nova] [tempest] [ceilometer] Defining Diagnotics API Schema (was: looking for clarification...)
On Tue, Oct 15, 2013 at 8:13 AM, Sean Dague s...@dague.net wrote: Open Ended API isn't really an API. An API has a contract, otherwise it isn't an API. I need that saying on a t-shirt. The diagnostics REST call currently seems to have no contract at all, and is just implemented by the underlying driver to whatever seems like a good idea today (also, no versioning on things, so what libvirt returns in grizzly vs. havana is up in the air). At minimum we need some kind of per driver (probably with a version) json schema so you could at least identify over the wire what you should be expecting. Even better would be a generic definition, but I get that takes time / effort. We have found it very useful in ceilometer to define classes (even simple namedtuples) for passing data into and out of our drivers. It is much easier to understand the expectations on both sides of the plugin API than when using dictionaries. I think the approach at this point is a bug on nova has been filed - https://bugs.launchpad.net/nova/+bug/1240043 ... and we're going to skip the tempest tests entirely based on that bug. Testing an API, that isn't actually an API, is really beyond tempest scope. On Mon, Oct 14, 2013 at 4:54 AM, Bob Ball bob.b...@citrix.com wrote: I’m happy with that approach – again I’ve not seen any discussions about how this should be done. ** ** I’ve added [tempest] and [ceilometer] tags so we can hopefully get input from the guys involved. ** ** Bob ** ** *From:* Gary Kotton [mailto:gkot...@vmware.com] *Sent:* 13 October 2013 05:21 *To:* OpenStack Development Mailing List *Subject:* Re: [openstack-dev] [nova] Looking for clarification on the diagnostics API ** ** Hi, I agree with Matt here. This is not broad enough. One option is to have a tempest class that overrides for various backend plugins. Then the test can be haredednd for each driver. I am not sure if that is something that has been talked about. Thanks Gary ** ** *From: *Matt Riedemann mrie...@us.ibm.com *Reply-To: *OpenStack Development Mailing List openstack-dev@lists.openstack.org *Date: *Sunday, October 13, 2013 6:13 AM *To: *OpenStack Development Mailing List openstack-dev@lists.openstack.org *Subject: *Re: [openstack-dev] [nova] Looking for clarification on the diagnostics API ** ** There is also a tempest patch now to ease some of the libvirt-specific keys checked in the new diagnostics tests there: https://review.openstack.org/#/c/51412/ To relay some of my concerns that I put in that patch: *I'm not sure how I feel about this. It should probably be more generic but I think we need more than just a change in tempest to enforce it, i.e. we should have a nova patch that changes the doc strings for the abstract compute driver method to specify what the minimum keys are for the info returned, maybe a doc api sample change, etc?* *For reference, here is the mailing list post I started on this last week:* * http://lists.openstack.org/pipermail/openstack-dev/2013-October/016385.html *http://lists.openstack.org/pipermail/openstack-dev/2013-October/016385.html *There are also docs here (these examples use xen and libvirt):* * http://docs.openstack.org/grizzly/openstack-compute/admin/content/configuring-openstack-compute-basics.html *http://docs.openstack.org/grizzly/openstack-compute/admin/content/configuring-openstack-compute-basics.html *And under procedure 4.4 here:* * http://docs.openstack.org/admin-guide-cloud/content/ch_introduction-to-openstack-compute.html#section_manage-the-cloud *http://docs.openstack.org/admin-guide-cloud/content/ch_introduction-to-openstack-compute.html#section_manage-the-cloud = I also found this wiki page related to metering and the nova diagnostics API: https://wiki.openstack.org/wiki/EfficientMetering/FutureNovaInteractionModel So it seems like if at some point this will be used with ceilometer it should be standardized a bit which is what the Tempest part starts but I don't want it to get lost there. Thanks, *MATT RIEDEMANN* Advisory Software Engineer Cloud Solutions and OpenStack Development -- *Phone:* 1-507-253-7622 | *Mobile:* 1-507-990-1889* E-mail:* mrie...@us.ibm.com [image: IBM] 3605 Hwy 52 N Rochester, MN 55901-1407 United States From:Gary Kotton gkot...@vmware.com To:OpenStack Development Mailing List openstack-dev@lists.openstack.org, Date:10/12/2013 01:42 PM Subject:Re: [openstack-dev] [nova] Looking for clarification on the diagnostics API -- Yup, it seems to be hypervisor specific. I have added in the Vmware support following you correcting in the Vmware driver. Thanks Gary *From: *Matt Riedemann mrie...@us.ibm.com* Reply-To: *OpenStack Development Mailing List
