Salvatore,

Thank you for your input. I actually took your suggestion about the area to 
cover for the CI systems and I updated the wiki page for the third party 
testing in Neutron.
https://wiki.openstack.org/wiki/NeutronThirdPartyTesting

Cheers,

Edgar

From: Salvatore Orlando <sorla...@nicira.com<mailto:sorla...@nicira.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Friday, August 15, 2014 at 4:11 PM
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

VMware minesweeper has filters which have been designed to cover the largest 
possible subset of submissions without add unnecessary load to our scarce 
resources for CI validation. This is probably why the analysis reveals not all 
patches are covered.

Therefore our filters exclude neutral changes such as those in README.rst 
(they're unlikely to crash neutron are they?), or changes in wsgi.py. The 
latter is because the wsgi framework is shared with all plugins, and when it 
breaks the NSX plugin it's very likely upstream tests will break too.

I think the areas which are important to cover are:
- neutron/agents/.* (or at least the agents you use)
- neutron/openstack/common/.*
- neutron/notifiers/.* (if your drivers report vif plugging events to nova)
- neutron/db/.*
- neutron/services/${thepluginsyouruninyourci_eveniftheyrenotyours}
- your plugin patch

Salvatore

PS: Openstack proposal bot submission's checks have been suspended for vmware 
minesweeper about 2 days ago. They will resume soon.


On 16 August 2014 01:02, Edgar Magana 
<edgar.mag...@workday.com<mailto:edgar.mag...@workday.com>> wrote:
Ivar,

Very valid point. This is why I need clarification from those CI owner.
I will run a new test with a basic change in the Neutron DB code. That should 
be covered by almost all CI systems.

Edgar

From: Ivar Lazzaro <ivarlazz...@gmail.com<mailto:ivarlazz...@gmail.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Friday, August 15, 2014 at 3:47 PM
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Hi Edgar,

Nice shot, to be the inquisitor is not necessarily a bad thing :)

I know some CIs are 'stuck' waiting for bugs to be fixed or certain patches to 
be merged, but I was wondering if it is a requirement that CIs vote *ALL* the 
Neutron patches. Some may have missed your call just because of their filters 
(see [0] section 'what changes to run against').

Cheers,
Ivar.

[0] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting




On Sat, Aug 16, 2014 at 12:35 AM, Edgar Magana 
<edgar.mag...@workday.com<mailto:edgar.mag...@workday.com>> wrote:
Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor here҆ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, "Kyle Mestery" 
<mest...@mestery.com<mailto:mest...@mestery.com>> wrote:

>Folks, I'm not sure if all CI accounts are running sufficient tests.
>Per the requirements wiki page here [1], everyone needs to be running
>more than just Tempest API tests, which I still see most neutron
>third-party CI setups doing. I'd like to ask everyone who operates a
>third-party CI account for Neutron to please look at the link below
>and make sure you are running appropriate tests. If you have
>questions, the weekly third-party meeting [2] is a great place to ask
>questions.
>
>Thanks,
>Kyle
>
>[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
>[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
>
>_______________________________________________
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to