On Thu, May 31, 2012 at 12:40 PM, Sanne Grinovero <sa...@infinispan.org> wrote: > On 31 May 2012 10:33, Galder Zamarreño <gal...@redhat.com> wrote: >> >> On May 29, 2012, at 12:48 PM, Manik Surtani wrote: >> >>> I pretty much agree with this; and here's a bit of history. >>> >>> For the large part we have had a stable test suite, but the occasional >>> unpredictability in the suite came in when we introduced the parallel test >>> runner, to allow us to run the (core) suite in under 5 minutes - a suite >>> which otherwise took over 2 hours when run sequentially. >>> >>> We could revert back to just using the sequential test runner if people >>> prefer that - it makes the suite run more predictably and hence easier to >>> debug and maintain - but the drawback is, well, it takes 2 hours to run. >>> >>> Perhaps we should use the parallel suite as a "smoke test", but in the >>> event of any failures, revert to a run using the sequential suite? >> >> I did some thinking on this and here's my view: >> >> There're tests that sometimes are sensitive to thread scheduling. We all >> have pretty powerful machines and often we won't see these issues, but when >> we go in CI, we might see them. >> >> What happens is that CI often uses machines that are less powerful, and if >> running a paralell testsuite in less powerful machines, these thread >> scheduling errors can come up more often. >> >> One way to solve this would be for individuals to run the testsuite in >> paralell, and when we go in CI, run it sequentially. >> >> This is what JDG is doing and trust me, it can highlight different issues in >> our test suite (seen it already), but at least, the thread scheduling issues >> are less common. >> >> Thoughts? > > I also thought it was a matter of "powerful", but the opposite way ;-) > Tests fail very often to me, much worse than on CI, and I always ended > up blaming the more powerful environment I'm running them in. > > FYI I'm unable to complete any test run on master, testsuite just > hangs. When it succeeds, I always had more failures than what I see on > CI: even while CI has been occasionally happy (like in February), I > never had a fully stable build, even when running it on a single > thread. > > Sorry if that wasn't clear, maybe I should complain about it more often :-P > > Seriously, I just think we can design the tests better. >
Do complain more often :) I got used to running only the core test suite all the time - I rely on the Jenkins results to check that I haven't broken everything else. > @Dan > sure you can reconfigure the CDI pom.xml to override the surefire settings. > Adi checked at the CDI pom.xml and surefire actually *was* configured to run tests sequentially. I guess the setting was just ignored for a a few builds - now it's back to running in one thread and there are no CDI tests failing in the last build. Cheers Dan _______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
