On 31 May 2012 10:33, Galder Zamarreño <gal...@redhat.com> wrote: > > On May 29, 2012, at 12:48 PM, Manik Surtani wrote: > >> I pretty much agree with this; and here's a bit of history. >> >> For the large part we have had a stable test suite, but the occasional >> unpredictability in the suite came in when we introduced the parallel test >> runner, to allow us to run the (core) suite in under 5 minutes - a suite >> which otherwise took over 2 hours when run sequentially. >> >> We could revert back to just using the sequential test runner if people >> prefer that - it makes the suite run more predictably and hence easier to >> debug and maintain - but the drawback is, well, it takes 2 hours to run. >> >> Perhaps we should use the parallel suite as a "smoke test", but in the event >> of any failures, revert to a run using the sequential suite? > > I did some thinking on this and here's my view: > > There're tests that sometimes are sensitive to thread scheduling. We all have > pretty powerful machines and often we won't see these issues, but when we go > in CI, we might see them. > > What happens is that CI often uses machines that are less powerful, and if > running a paralell testsuite in less powerful machines, these thread > scheduling errors can come up more often. > > One way to solve this would be for individuals to run the testsuite in > paralell, and when we go in CI, run it sequentially. > > This is what JDG is doing and trust me, it can highlight different issues in > our test suite (seen it already), but at least, the thread scheduling issues > are less common. > > Thoughts?
I also thought it was a matter of "powerful", but the opposite way ;-) Tests fail very often to me, much worse than on CI, and I always ended up blaming the more powerful environment I'm running them in. FYI I'm unable to complete any test run on master, testsuite just hangs. When it succeeds, I always had more failures than what I see on CI: even while CI has been occasionally happy (like in February), I never had a fully stable build, even when running it on a single thread. Sorry if that wasn't clear, maybe I should complain about it more often :-P Seriously, I just think we can design the tests better. @Dan sure you can reconfigure the CDI pom.xml to override the surefire settings. Cheers, Sanne _______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
