it should take 1 hour and half. Maybe we could fix some issues on the tests and make it to 1 hour.
Up to now we have been taking good care on avoiding sleeps and making sure the tests run without delays. There are a few tests that will need to validate expiration and etc so it's unavoidable a few sleeps (like failover for instance). A lot of time on the testsuite is being spent on Failover and Bridge Tests.. we could make them running faster without losing coverage. There is also the stress-testsuite and soak tests but those are basically an extension of the testsuite (increasing number of iterations in some of the tests); we will eventually make that part of the regression builds.. but we should make that less frequent (once a week and always before releases). The testsuite now is not working as I broke the testsuite in some of my last commits (I'm trying to fix that now), but this should give you an idea of what we run: https://builds.apache.org/view/All/job/ActiveMQ6-Nightly-Regression-Test/68/testReport/ I should have it fixed after the holidays (working on it) On Thu, Apr 2, 2015 at 9:22 PM, Kevin Burton <[email protected]> wrote: > NICE. Also, how long does the entire test / regression suite take? … > roughly. > > On Wed, Apr 1, 2015 at 1:01 PM, Clebert Suconic <[email protected]> > wrote: > >> For every Pull Request sent (e.g: >> https://github.com/apache/activemq-6/pull/197) >> >> There will be a build with a considerable sized testsuite running.. >> which we call fast-test >> https://builds.apache.org/job/ActiveMQ6-PR-Build/223/ >> >> >> This will validate tests, CheckStyle and Licensing. >> >> >> >> This will run the entire JMS tests, unit-tests and some other tests. >> It's worth looking at: >> https://builds.apache.org/job/ActiveMQ6-PR-Build/223/testReport/ >> >> >> >> We then run the entire testsuite daily: >> https://builds.apache.org/view/All/job/ActiveMQ6-Nightly-Regression-Test/ >> >> >> There are some failures on this environment now but that's something >> we are working to solve with Infra. They run ok on my personal dev >> enrionment and some other servers I have access to. >> >> We are in hold now based on the discussions on dev-list this week, but >> we should fix that soon. >> >> >> Anyway, I hope you get the picture.. PRs are rejected if they fail the >> basic testsuite, and we will monitor the entire testsuite daily. >> >> On Wed, Apr 1, 2015 at 3:48 PM, Kevin Burton <[email protected]> wrote: >> > This was one of the things that bothered/bothers me about ActiveMQ 5.x… >> the >> > test takes 24 hours to run. I’ve never actually had any success getting >> it >> > to run because it’s crashed on me every time. (even for release builds). >> > >> > Our internal tests take 20 minutes to build… and that is too long IMO. >> > Certainly our tests could be more comprehensive but I think they’re >> pretty >> > damn good right now. >> > >> > Would be good if ActiveMQ 6.0 could resolve this and we could get builds >> > integrated quickly after a commit. :) >> > >> > Kevin >> > >> > -- >> > >> > Founder/CEO Spinn3r.com >> > Location: *San Francisco, CA* >> > blog: http://burtonator.wordpress.com >> > … or check out my Google+ profile >> > <https://plus.google.com/102718274791889610666/posts> >> > <http://spinn3r.com> >> >> >> >> -- >> Clebert Suconic >> http://community.jboss.org/people/[email protected] >> http://clebertsuconic.blogspot.com >> > > > > -- > > Founder/CEO Spinn3r.com > Location: *San Francisco, CA* > blog: http://burtonator.wordpress.com > … or check out my Google+ profile > <https://plus.google.com/102718274791889610666/posts> > <http://spinn3r.com> -- Clebert Suconic http://community.jboss.org/people/[email protected] http://clebertsuconic.blogspot.com
