> jenkins "run all tests continuously" build target that only runs the
> @AwaitsFix group, and overrides haltonfailure when calling the junit
> macro. (which would save us an extra jenkins run)
Yep, doable.
> decreased the likelyhood of failure, but didn't completely fix the
> problem -- a test m
: This is doable by enabling/disabling test groups. A new build plan
: would need to be created that would do:
:
: ant -Dtests.haltonfailure=false -Dtests.awaitsfix=true
: -Dtests.unstable=true test
right ...that's an idea that came up the other day when i was talking to
simon at revolution
an
> That was really the main question i had, as someone not very familiar with
> the internals of JUnit, is wether it was possible for our test runner to
> make the ultimate decision about the success/fail status of the entire
> run based on the annotations of the tests that fail/succed
There are tw
: as a build artifact). Yet another problem is that jenkins wouldn't
: _fail_ on such pseudo-failures because the set of JUnit statuses is
: not extensible (it'd be something like FAILED+IGNORE) so we'd need to
That was really the main question i had, as someone not very familiar with
the intern
On Sun, May 6, 2012 at 3:38 PM, Dawid Weiss
wrote:
> I also admit I've never seen anything like
> this -- a suite of tests with an allowed failure ratio over time and a
> threshold that would trigger a warning...
Not so much an "allowed" failure rate... more of "it fails sometimes
and no one has
> If the tests are run many times a day, it would be great to get a
> daily report of the percent of time the tests pass. Then if it goes
> from 5% to 50%, we can go uh-oh...
Yeah, well... but this is beyond the runner as it aggregates over time
-- it looks like a jenkins plugin that would analyz
On Sun, May 6, 2012 at 2:39 PM, Dawid Weiss
wrote:
> Any ideas? Hoss -- how do you envision "monitoring" of these tests? Manually?
If the tests are run many times a day, it would be great to get a
daily report of the percent of time the tests pass. Then if it goes
from 5% to 50%, we can go uh-oh
So, I started thinking about it -- I can implement something that will
report failures (much like we do right now) it's quite tricky to fit
it into the reporting system and continuous integration system. Here's
why -- if a test doesn't fail then its output (sysout/syserrs) are not
currently printed
One other way to do it that is already implemented is to run full tests
without failing on failures and only touch some marker file to fail at the
end. Ant test-help gives a hint on how to run tests this way currently.
Dont know how it'd play with others - speak up.
On May 5, 2012 11:25 AM, "Mikhai
It's a pretty useful technique especially from CI perspective. I use it via
JUnit's assumptions. Failed assumption is shown as an ignored test.
On Sat, May 5, 2012 at 2:29 AM, Chris Hostetter wrote:
>
> Dawid:
>
> With the new test runner you created, would it be possible to setup an
> annotation
Not a problem. Ill be at work on monday. Can you file an issue an assign it
to me please? Im currently on mobile only. Dawid
On May 5, 2012 5:08 AM, "Yonik Seeley" wrote:
> On Fri, May 4, 2012 at 6:29 PM, Chris Hostetter
> wrote:
> >
> > Dawid:
> >
> > With the new test runner you created, would
On Fri, May 4, 2012 at 6:29 PM, Chris Hostetter
wrote:
>
> Dawid:
>
> With the new test runner you created, would it be possible to setup an
> annotation that we could use instead to indicate that a test should in fact
> be run, and if it fails, include the failure info in the build report, but
>
Dawid:
With the new test runner you created, would it be possible to setup an
annotation that we could use instead to indicate that a test should in
fact be run, and if it fails, include the failure info in the build
report, but do not fail the build?
I'm thinking in particular about some o
13 matches
Mail list logo