Wow - that looks really useful :-) I wonder if it could use the generic hudson.task.test.* generic base classes rather than the hudson.task.junit ones so that it would work with other test data publishers... If not looks like it would be pretty simple to wrap other test publishers in the same manner!
BTW - would be helpfully if you could put a LICENCE.txt in the root folder, or in the source files. Otherwise some people may not be able to contribute fixes. /James On Thursday, 12 September 2013 03:21:47 UTC+1, Samuel Nobs wrote: > > hi all, > > i have come up with a plugin to quarantine flakey tests. i first intended > to merge it into the claim-plugin, but then diverged more and more both > conceptually and from an implementation point of view. therefore i think > it's more appropriate to have it as a separate plugin. > > see also: https://issues.jenkins-ci.org/browse/JENKINS-14089 > > i have currently hosted it on: > https://github.com/samsta/quarantine > > > here's a quick description: > > imagine you have a flakey test that sometimes passes, sometimes fails. if > you go to the latest result of the test, that's what you see: > [image: Inline image 1] > let's quarantine it: > [image: Inline image 2] > after hitting quarantine: > [image: Inline image 3] > and on the test result page one level up: > > [image: Inline image 5] > > this won't make the job go green immediately, as jenkins does not allow > you to change the build status after the fact. but on the next run, the > following will happen (from the console output): > > Recording test results > [Quarantine]: AnotherSuiteB.TestA failed but is quarantined > [Quarantine]: 0 unquarantined failures remaining > Finished: SUCCESS > > if now another test fails that has not been quarantined, this happens: > > Recording test results > [Quarantine]: AnotherSuiteB.TestA failed but is quarantined > [Quarantine]: 1 unquarantined failures remaining > Build step 'Publish quarantinable JUnit test result report' changed build > result to UNSTABLE > Finished: UNSTABLE > > there's also a top-level page listing all the tests that are in quarantine > so we don't forget about those quarantined tests: > > [image: Inline image 6] > > > cheers, > sam > -- > Samuel Nobs > Senior Design Engineer > Tait Communications > DDI: +64 7 825 7538 > Email: [email protected] <javascript:> > > <http://www.taitradio.com> > www.taitradio.com > > > > > > -- > Samuel Nobs > Senior Design Engineer > Tait Communications > DDI: +64 7 825 7538 > Email: [email protected] <javascript:> > > <http://www.taitradio.com> > www.taitradio.com > > > ------------------------------ > This email, including any attachments, is only for the intended recipient. > It is subject to copyright, is confidential and may be the subject of legal > or other privilege, none of which is waived or lost by reason of this > transmission. > If you are not an intended recipient, you may not use, disseminate, > distribute or reproduce such email, any attachments, or any part thereof. > If you have received a message in error, please notify the sender > immediately and erase all copies of the message and any attachments. > Unfortunately, we cannot warrant that the email has not been altered or > corrupted during transmission nor can we guarantee that any email or any > attachments are free from computer viruses or other conditions which may > damage or interfere with recipient data, hardware or software. The > recipient relies upon its own procedures and assumes all risk of use and of > opening any attachments. > ------------------------------ > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
