On 10/11/2013 02:03 PM, Alessandro Pilotti wrote:




On Oct 11, 2013, at 19:29 , Russell Bryant <rbry...@redhat.com <mailto:rbry...@redhat.com>>
 wrote:

On 10/11/2013 12:04 PM, John Griffith wrote:

[... snip ...]

Talking about new community involvements, newcomers are getting very frustrated to have to wait for weeks to get a meaningful review and I cannot blame them if they don't want to get involved anymore after the first patch! This makes appear public bureocracy here in eastern Europe a lightweight process in comparison! :-)

Let me add another practical reason about why a separate OpenStack project would be a good idea:

Anytime that we commit a driver specific patch, a lot of Tempests tests are executed on Libvirt and XenServer (for Icehouse those will be joined by another pack of CIs, including Hyper-V). On the jenkins side, we have to wait for regression tests that have nothing to do with the code that we are pushing. During the H3 push, this meant waiting for hours and hoping not to have to issue the 100th "recheck / revery bug xxx".

A separate project would obviously include only the required tests and be definitely more lightweight, offloading quite some work from the SmokeStack / Jenkins job for everybody's happiness.


I'm glad you brought this up. There are two issues here, both discussed by the qe/infra groups and others at the Havana summit and after.

How do you/we know which regression tests have nothing to do with the code changed in a particular patch? Or that the answer won't change tomorrow? The only way to do that is to assert dependencies and non-dependencies between components that will be used to decide which tests should be run for each patch. There was a lively discussion (with me taking your side initially) at the summit and it was decided that a generic "wasting resources" argument was not sufficient to introduce that fragility and so we would run the whole test suite as a gate on all projects. That decision was to be revisited if resources became a problem.

As for the 100th recheck, that is a result of the recent introduction of parallel tempest runs before the Havana rush. It was decided that the benefit in throughput from drastically reduced gate job times outweighed the pain of potentially doing a lot of rechecks. For the most part the bugs being surfaced were real OpenStack bugs that were showing up due to the new "stress" of parallel test execution. This was a good thing, though certainly painful to all. With hindsight I'm not sure if that was the right decision or not.

This is just an explanation of what has happened and why. There are obviously costs and benefits of being tightly bound to the project.

 -David
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to