Hey Everyone, A while back I started talking about this idea of requiring Cinder driver contributors to run a super simple cert script (some info here: [1]). Since then I've been playing with introduction of a third party gate check here in my own lab. My proposal was to have a non-voting check that basically duplicates the base devstack gate test in my lab, but uses different back-end devices that I have available configured in Cinder to run periodic tests against. Long term I'd like to be able to purpose this gear to also do something "more useful" for the over all OpenStack gating effort but to start it's strictly an automated verification of my Cinder driver/backend.
What I'm questioning is how to report this information and the results. Currently patches and reviews are our mechanism for triggering tests and providing feedback. Myself and many other vendors that might like to participate in something like this obviously don't have the infrastructure to try and run something like this on every single commit. Also since it would be non-voting it's difficult to capture and track the results. One idea that I had was to set something like what I've described above to run locally on a periodic basis (weekly, nightly etc) and publish results to something like a "third party verification dashboard". So the idea would be that results from various third party tests would all adhere to a certain set of criteria WRT what they do and what they report and those results would be logged and tracked publicly for anybody in the OpenStack community to access and view? Does this seem like something that others would be interested in participating in? I think it's extremely valuable for projects like Cinder that have dozens of backend devices, and regardless of other interest or participation in the community I intend to implement something like this on my own regardless. It would just be interesting to see if we could have an organized and official effort to gather this sort of information and run these types of tests. Open to suggestions and thoughts as well as any of you that may already be doing this sort of thing. By the way, I've been looking at things like SmokeStack and other third party gating checks to get some ideas as well. Thanks, John _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev