Salvatore, Thank you for your input. I actually took your suggestion about the area to cover for the CI systems and I updated the wiki page for the third party testing in Neutron. https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
Cheers, Edgar From: Salvatore Orlando <sorla...@nicira.com<mailto:sorla...@nicira.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Friday, August 15, 2014 at 4:11 PM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to be run VMware minesweeper has filters which have been designed to cover the largest possible subset of submissions without add unnecessary load to our scarce resources for CI validation. This is probably why the analysis reveals not all patches are covered. Therefore our filters exclude neutral changes such as those in README.rst (they're unlikely to crash neutron are they?), or changes in wsgi.py. The latter is because the wsgi framework is shared with all plugins, and when it breaks the NSX plugin it's very likely upstream tests will break too. I think the areas which are important to cover are: - neutron/agents/.* (or at least the agents you use) - neutron/openstack/common/.* - neutron/notifiers/.* (if your drivers report vif plugging events to nova) - neutron/db/.* - neutron/services/${thepluginsyouruninyourci_eveniftheyrenotyours} - your plugin patch Salvatore PS: Openstack proposal bot submission's checks have been suspended for vmware minesweeper about 2 days ago. They will resume soon. On 16 August 2014 01:02, Edgar Magana <edgar.mag...@workday.com<mailto:edgar.mag...@workday.com>> wrote: Ivar, Very valid point. This is why I need clarification from those CI owner. I will run a new test with a basic change in the Neutron DB code. That should be covered by almost all CI systems. Edgar From: Ivar Lazzaro <ivarlazz...@gmail.com<mailto:ivarlazz...@gmail.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Friday, August 15, 2014 at 3:47 PM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to be run Hi Edgar, Nice shot, to be the inquisitor is not necessarily a bad thing :) I know some CIs are 'stuck' waiting for bugs to be fixed or certain patches to be merged, but I was wondering if it is a requirement that CIs vote *ALL* the Neutron patches. Some may have missed your call just because of their filters (see [0] section 'what changes to run against'). Cheers, Ivar. [0] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting On Sat, Aug 16, 2014 at 12:35 AM, Edgar Magana <edgar.mag...@workday.com<mailto:edgar.mag...@workday.com>> wrote: Team, I did a quick audit on the Neutron CI. Very sad results. Only few plugins and drivers are running properly and testing all Neutron commits. I created a report here: https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin _and_Drivers We will discuss the actions to take on the next Neutron IRC meeting. So please, reach me out to clarify what is the status of your CI. I had two commits to quickly verify the CI reliability: https://review.openstack.org/#/c/114393/ https://review.openstack.org/#/c/40296/ I would expect all plugins and drivers passing on the first one and failing for the second but I got so many surprises. Neutron code quality and reliability is a top priority, if you ignore this report that plugin/driver will be candidate to be remove from Neutron tree. Cheers, Edgar P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job! On 8/14/14, 8:30 AM, "Kyle Mestery" <mest...@mestery.com<mailto:mest...@mestery.com>> wrote: >Folks, I'm not sure if all CI accounts are running sufficient tests. >Per the requirements wiki page here [1], everyone needs to be running >more than just Tempest API tests, which I still see most neutron >third-party CI setups doing. I'd like to ask everyone who operates a >third-party CI account for Neutron to please look at the link below >and make sure you are running appropriate tests. If you have >questions, the weekly third-party meeting [2] is a great place to ask >questions. > >Thanks, >Kyle > >[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting >[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty > >_______________________________________________ >OpenStack-dev mailing list >OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
