Excerpts from Chris Hoge's message of 2016-06-14 10:57:05 -0700: > Last year, in response to Nova micro-versioning and extension updates[1], > the QA team added strict API schema checking to Tempest to ensure that > no additional properties were added to Nova API responses[2][3]. In the > last year, at least three vendors participating the the OpenStack Powered > Trademark program have been impacted by this change, two of which > reported this to the DefCore Working Group mailing list earlier this year[4]. > > The DefCore Working Group determines guidelines for the OpenStack Powered > program, which includes capabilities with associated functional tests > from Tempest that must be passed, and designated sections with associated > upstream code [5][6]. In determining these guidelines, the working group > attempts to balance the future direction of development with lagging > indicators of deployments and user adoption. > > After a tremendous amount of consideration, I believe that the DefCore > Working Group needs to implement a temporary waiver for the strict API > checking requirements that were introduced last year, to give downstream > deployers more time to catch up with the strict micro-versioning > requirements determined by the Nova/Compute team and enforced by the > Tempest/QA team. > > My reasoning behind this is that while the change that enabled strict > checking was discussed publicly in the developer community and took > some time to be implemented, it still landed quickly and broke several > existing deployments overnight. As Tempest has moved forward with > bug and UX fixes (some in part to support the interoperability testing > efforts of the DefCore Working Group), using an older versions of Tempest > where this strict checking is not enforced is no longer a viable solution > for downstream deployers. The TC has passed a resolution to advise > DefCore to use Tempest as the single source of capability testing[7], > but this naturally introduces tension between the competing goals of > maintaining upstream functional testing and also tracking lagging > indicators. > > My proposal for addressing this problem approaches it at two levels: > > * For the short term, I will submit a blueprint and patch to tempest that > allows configuration of a grey-list of Nova APIs where strict response > checking on additional properties will be disabled. So, for example, > if the 'create servers' API call returned extra properties on that call, > the strict checking on this line[8] would be disabled at runtime. > Use of this code path will emit a deprecation warning, and the > code will be scheduled for removal in 2017 directly after the release > of the 2017.01 guideline. Vendors would be required so submit the > grey-list of APIs with additional response data that would be > published to their marketplace entry. > > * Longer term, vendors will be expected to work with upstream to update > the API for returning additional data that is compatible with > API micro-versioning as defined by the Nova team, and the waiver would > no longer be allowed after the release of the 2017.01 guideline. > > For the next half-year, I feel that this approach strengthens interoperability > by accurately capturing the current state of OpenStack deployments and > client tools. Before this change, additional properties on responses > weren't explicitly disallowed, and vendors and deployers took advantage > of this in production. While this is behavior that the Nova and QA teams > want to stop, it will take a bit more time to reach downstream. Also, as > of right now, as far as I know the only client that does strict response > checking for Nova responses is the Tempest client. Currently, additional > properties in responses are ignored and do not break existing client > functionality. There is currently little to no harm done to downstream > users by temporarily allowing additional data to be returned in responses.
Thanks for putting this proposal together, Chris. The configuration option you describe makes sense as a temporary solution to the issue, and the timeline you propose (combined with the past year since the change went in) should be plenty of time to handle upgrades. Doug > > Thanks, > > Chris Hoge > Interop Engineer > OpenStack Foundation > > [1] > https://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/api-microversions.html > [2] > http://lists.openstack.org/pipermail/openstack-dev/2015-February/057613.html > [3] https://review.openstack.org/#/c/156130 > [4] > http://lists.openstack.org/pipermail/defcore-committee/2016-January/000986.html > [5] http://git.openstack.org/cgit/openstack/defcore/tree/2015.07.json > [6] http://git.openstack.org/cgit/openstack/defcore/tree/2016.01.json > [7] > http://git.openstack.org/cgit/openstack/governance/tree/resolutions/20160504-defcore-test-location.rst > [8] > http://git.openstack.org/cgit/openstack/tempest-lib/tree/tempest_lib/api_schema/response/compute/v2_1/servers.py#n39 > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
