If that's what you want, then that's a good thing to do. I prefer having "Unstable" mean "code compiled, one or more tests failed" so I gained more meaning from the "red" / "yellow" / "blue" states of the build. If you gain more meaning by having two states, "red" and "blue" for your results, that's OK too.
I had not considered the case where you truly have no compilation phase at all, since all the projects I use involve a mixture of compiled and dynamic languages, with the dynamic languages typically not being invoked until after the compilation phase is complete. The cases where I've needed to run MSTest (a few years ago now) also required that I be able to process more than one MSTest output file at a time, and that capability was not available at that time (added in version 0.7). I see that the capability to process multiple files with wildcards has been available for over a year, so I'm glad it meets your needs. Using the MSTest plugin seems like a good way to do it. I think the idea of causing your MSTest invocation to return a non-zero exit code is still a reasonable approach to give you the failure you're seeking when a test fails. You might also consider using the MSTestRunner plugin to launch MSTest ( https://wiki.jenkins-ci.org/display/JENKINS/MSTestRunner+Plugin ) to see if it has a way to mark the job as failed when a test fails. Yet another alternative would be to investigate the xUnit plugin to see if it will give you the flexibility you want. Mark Waite >________________________________ > From: bearrito <j.barrett.straus...@gmail.com> >To: jenkinsci-users@googlegroups.com >Cc: Mark Waite <markwa...@yahoo.com> >Sent: Friday, September 7, 2012 11:35 AM >Subject: Re: MSTest Plugin doesn't fail the job on failing test. > > >Not marking a build as failed with failing unit tests doesn't make any sense >to me. We fail the build if a compile fails so why not with tests? >What if I was working in a dynamic/interpreted language and so couldn't rely >on compiler/type system to find what would be guaranteed run-time errors? > > >I'd prefer to use the MSTest plugin because the MSTest plugin performs the >conversion from the .trx test output to the junit format that jenkins >understands. > > >-barrett > >On Thursday, September 6, 2012 1:51:00 PM UTC-4, Mark Waite wrote: >You might consider the comments in this thread before you mark a job as failed >due to test failures: >> >> >>http://jenkins.361315.n4. nabble.com/plugin-A-build- >>with-failed-tests-should-be- marked-as-Unstable-or-Failure- td3585663.html >> >> >> >>If that guidance doesn't meet your needs and you decide that you still want >>to fail the build on test failures rather than mark the build "Unstable" on >>test failures, then I think the easiest way is to invoke mstest.exe directly >>(instead of using the MSTest plugin to launch it) and then have the MSTest >>batch file exit with a non zero exit code if any test fails. >> >> >>Mark Waite >> >> >> >> >> >> >> >>>________________________________ >>> From: bearrito <j.barrett...@gmail.com > >>>To: jenkins...@googlegroups. com >>>Sent: Thursday, September 6, 2012 10:04 AM >>>Subject: MSTest Plugin doesn't fail the job on failing test. >>> >>> >>>I would like the MSTest plugin to fail the job when a test fails. Is this >>>feasible ? >>> >>> >>> >>> >>> >>> >>> >>> > >
