On May 30, 2012, at 3:25 PM, Dan Berindei wrote: > On Wed, May 30, 2012 at 4:12 PM, Manik Surtani <ma...@jboss.org> wrote: >> >> On 30 May 2012, at 13:55, Sanne Grinovero wrote: >> >>> On 30 May 2012 13:36, Manik Surtani <ma...@jboss.org> wrote: >>>> Well, one problem leads to another, as you are well aware. Flaky parallel >>>> suite leads careless commits. Yes, we should fix what is broken at the >>>> moment but that is not enough since it will get this way again unless we >>>> have a stable suite that can be used to ensure quality moving fwd. The >>>> options are: >>>> >>>> 1. We always use the sequential suite. >>>> 2. We identify tests that are reliable with the parallel suite (should be >>>> at least 90% of the tests IMO) and use this, leaving the sequential suite >>>> to BuildHive. >>> >>> Ah, now I understand. You want to have some tests executed only when >>> run sequentially.. that looks like a nice compromise, cheers! >>> that will also speed up our test runs, as not all tests are executed; >>> we could exclude also some of the slower ones even if they could work >>> fine in parallel. >> >> Precisely. No loss in quality, just changing the order of delivery of >> information to the reviewer. :) >> > > Ok, first on the list of sequential-only tests: everything that uses > Arquillian :) > I was looking at the cdi test failures and it looks like Arquillian > uses a thread-local variable and that is initialized only on the > thread that calls their @BeforeSuite method. So every time > Arquillian.arquillianBefore/AfterClass runs in a different thread, it > throws a NullPointerException. > > I see two issues with your plan, though: > 1. Buildhive is limited to 15 mins, and a reviewer wouldn't > necessarily wait for 2 hours to integrate a pull request anyway. So > the sequential build would be limited to Jenkins runs.
^ Just got a msg today that Buildhive has changed default timeout to 120 mins. I'll modify Buildhive soon (ish). > 2. How do we select which tests run where? I remember we had to > disable tests precisely because configuring test groups in > testng/surefire didn't work. > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache _______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
