On 04/22/2014 08:35 AM, Daniel P. Berrange wrote: > On Tue, Apr 22, 2014 at 07:23:20AM -0400, Sean Dague wrote: >> On 04/22/2014 06:55 AM, Daniel P. Berrange wrote: >>> On Mon, Apr 21, 2014 at 10:52:39PM +0000, Daryl Walleck wrote: >>>> I nearly opened a spec for this, but I'd really like to get some >>>> feedback first. One of the challenges I've seen lately for Nova >>>> teams not using KVM or Xen (Ironic and LXC are just a few) is >>>> how to properly run the subset of Compute tests that will run for >>>> their hypervisor or driver. Regexes are what Ironic went with, >>>> but I'm not sure how well that will work long term since it's >>>> very much dependent on naming conventions. The good thing is >>>> that the capabilities for each hypervisor/driver are well defined >>>> (https://wiki.openstack.org/wiki/HypervisorSupportMatrix), so >>>> it's just a matter of how to convey that information. I see a >>>> few ways forward from here: >>>> >>>> 1. Expand the compute_features_group config section to include >>>> all Compute actions and make sure all tests that require specific >>>> capabilities have skipIfs or raise a skipException. This options seems >>>> it would require the least work within Tempest, but the size of the >>>> config will continue to grow as more Nova actions are added. >>>> >>>> 2. Create a new decorator class like was done with service tags >>>> that defines what drivers the test does or does not work for, and have >>>> the definitions of the different driver capabilities be referenced by >>>> the decorator. This is nice because it gets rid of the config creep, >>>> but it's also yet another decorator, which may not be desirable. >>>> >>>> I'm going to continue working through both of these possibilities, >>>> but any feedback either solution would be appreciated. >>> >>> It strikes me that if the test suites have a problem determining >>> support status of APIs with the currently active driver, then the >>> applications using OpenStack will likely suffer the same problem. >>> Given that it'd be desirable to ensure we can solve it in a general >>> way, rather than only consider the test suite needs. >>> >>> On the Nova side of things, I think it would be important to ensure >>> that there is a single "OperationNotSupported" exception that is >>> always raised when the API tries to exercise a feature that is not >>> available with a specific hypervisor driver. If a test case in the >>> test suite ever receives "OperationNotSupported" it could then just >>> mark that test case as "skipped" rather than having exception propagate >>> to result in a "fail". To me the nice thing about such an approach is >>> that you do not need to ever maintain a "matrix" of supported features, >>> as the test suite would just do the right thing whenever the driver is >>> updated. >> >> Agreed. Though I think we probably want the Nova API to be explicit >> about what parts of the API it's ok to throw a Not Supported. Because I >> don't think it's a blanket ok. On API endpoints where this is ok, we can >> convert not supported to a skip. > > Yep, this ties into a discussion I recall elsewhere about specifying > exactly which parts of the Nova API are considered mandatory features > for drivers to implement.
I put that in as a proposal to discuss at the design summit. http://summit.openstack.org/cfp/details/55 -- Russell Bryant _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
