On Sun, Jan 19, 2014 at 2:47 PM, Devananda van der Veen < devananda....@gmail.com> wrote:
> Hi all, > > I've been thinking about how we should treat third-party drivers in Ironic > for a while, and had several discussions at the Hong Kong summit and last > week at LCA. Cinder, Nova, Neutron, and TripleO are all having similar > discussions, too. What follows is a summary of my thoughts and a proposal > for our project's guidelines to vendors. > I applaud the effort. I'm actually currently in the process of writing up instructions for Cinder and Neutron vendors interested in constructing a 3rd party testing platform that uses the openstack-infra tooling as much as possible. (Yes, I know there is existing documentation on ci.openstack.org, but based on discussions this past week with the Neutron vendors implementing these test platforms, there's a number of areas that are poorly understood and some more detail is clearly needed). I would hope the docs I'm putting together for Cinder and Neutron will require little, if any, changes for similar instructions for Ironic 3rd party testers. > Before requiring that degree of testing, I would like to be able to direct > vendors at a working test suite which they can copy. I expect us to have > functional testing for the PXE and SSH drivers within Tempest and devstack > / devstack-gate either late in this cycle or early next cycle. Around the > same time, I believe TripleO will switch to using Ironic in their test > suite, so we'll have coverage of the IPMI driver on real hardware as well > (this may be periodic coverage rather than per-test feedback initially). > I think using Tempest as that working test suite would be the best way to go. Cinder 3rd party testing is going in this direction (the cinder_cert/ directory in devstack simply sets up Tempest, sets the appropriate Cinder driver properly in the cinder.conf and then runs the Tempest Volume API tests. A similar approach would work for Ironic, I believe, once the Ironic API tests are complete for Tempest. > I am proposing that we provisionally allow in vendor drivers this cycle > with the following requirements, and that we draw the line at the J3 > milestone to give everyone ample time to get testing up on real hardware -- > without blocking innovation now. At that time, we may kick out third-party > drivers if these criteria haven't been met. > > 1. each driver must adhere to the existing driver interfaces. > 2. each driver must have comprehensive unit test coverage and sufficient > inline documentation. > 3. vendors are responsible for fixing bugs in their driver in a timely > fashion. > 4. vendors commit to have third-party testing on a supported hardware > platform implemented by the J3 milestone. > 5. vendors contribute a portion of at least one developer's time to > upstream participation. > All good things. However, specificity is critical here. What does "sufficient inline documentation" entail? Who is the arbiter? What does "comprehensive unit test coverage" mean? 90%? 100%? What does "timely fashion" mean? Within 2 days? By X milestone? The more specificity, the less miscommunication will occur. And, BTW, the above goes for all the driver verification programs currently being fleshed out... not just Ironic, of course! :) Best, -jay
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev