I've spent some time doing some initial analysis of 3rd Party CI in Neutron. The tl;dr is that it's a mess, and it needs fixing. And I'm setting a deadline of Juno-2 for people to address their CI systems and get them in shape or we will remove plugins and drivers in Juno-3 which do not meet the expectations set out below.
My initial analysis of Neutron 3rd Party CI is here [1]. This was somewhat correlated with information from DriverLog [2], which was helpful to put this together. As you can see from the list, there are a lot of CI systems which are broken right now. Some have just recently started working again. Others are working great, and some are in the middle somewhere. The overall state isn't that great. I'm sending this email to openstack-dev and BCC;ing CI owners to raise awareness of this issue. If I have incorrectly labeled your CI, please update the etherpad and include links to the latest voting/comments your CI system has done upstream and reply to this thread. I have documented the 3rd Party CI requirements for Neutron here [3]. I expect people to be following these guidelines for their CI systems. If there are questions on the guidelines or expectations, please reply to this thread or reach out to myself in #openstack-neutron on Freenode. There is also a third-party meeting [4] which is a great place to ask questions and share your experience setting up a 3rd party CI system. The infra team has done a great job sponsoring and running this meeting (thanks Anita!), so please both take advantage of it and also contribute to it so we can all share knowledge and help each other. Owners of plugins/drivers should ensure their CI is matching the requirements set forth by both infra and Neutron when running tests and posting results. Like I indicated earlier, we will look at removing code for drivers which are not meeting these requirements as set forth in the wiki pages. The goal of this effort is to ensure consistency across testing platforms, making it easier for developers to diagnose issues when third party CI systems fail, and to ensure these drivers are tested since they are part of the integrated releases we perform. We used to require a core team member to sponsor a plugin/driver, but we moved to the 3rd party CI system in Icehouse instead. Ensuring these systems are running and properly working is the only way we can ensure code is working when it's part of the integrated release. Thanks, Kyle [1] https://etherpad.openstack.org/p/ZLp9Ow3tNq [2] http://www.stackalytics.com/driverlog/?project_id=openstack%2Fneutron&vendor=&release_id= [3] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting [4] https://wiki.openstack.org/wiki/Meetings/ThirdParty _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev