Re: Disabling flaky tests

2013-06-05 Thread Bertrand Delacretaz
On Tue, Jun 4, 2013 at 6:37 PM, Carsten Ziegeler cziege...@apache.org wrote: ...I would agree if the jenkins we have would be stable; i have the feeling that a lot of build failures on that instance are simply because of some problems of the build server itself... There were issues in the last

RE: Disabling flaky tests

2013-06-04 Thread Jeff Young
. -Original Message- From: Carsten Ziegeler [mailto:cziege...@apache.org] Sent: 03 June 2013 07:01 To: dev@sling.apache.org Subject: Re: Disabling flaky tests I agree as well, especially for the error handling as this is partially not a problem of the test but really a bug

Re: Disabling flaky tests

2013-06-04 Thread Carsten Ziegeler
: Carsten Ziegeler [mailto:cziege...@apache.org] Sent: 03 June 2013 07:01 To: dev@sling.apache.org Subject: Re: Disabling flaky tests I agree as well, especially for the error handling as this is partially not a problem of the test but really a bug in Sling - we have an issue

Re: Disabling flaky tests

2013-06-03 Thread Carsten Ziegeler
I agree as well, especially for the error handling as this is partially not a problem of the test but really a bug in Sling - we have an issue for that, it just needs to be done :) Carsten 2013/6/3 Felix Meschberger fmesc...@adobe.com I agree here: Disabling the test and having an issue keeps

Re: Disabling flaky tests

2013-06-03 Thread Bertrand Delacretaz
Hi, On Fri, May 31, 2013 at 9:14 PM, Robert Munteanu romb...@apache.org wrote: ...It seems that the ErrorHandlingTest fails sporadically when run inside a full maven build... I fixed a related issue in SLING-2818 a few weeks ago, is it something else now? For this test, and for future

Re: Disabling flaky tests

2013-06-03 Thread Bertrand Delacretaz
On Mon, Jun 3, 2013 at 10:00 AM, Bertrand Delacretaz bdelacre...@apache.org wrote: ...I fixed a related issue in SLING-2818 a few weeks ago, is it something else now?... Looking at it again there might be some latency after uploading an error handler script, before it is activated. It's not a

Re: Disabling flaky tests

2013-06-03 Thread Robert Munteanu
Hm, my proposal isn't winning any popularity contests, so I won't go through with it :-)

Re: Disabling flaky tests

2013-06-03 Thread Robert Munteanu
On Mon, 2013-06-03 at 10:00 +0200, Bertrand Delacretaz wrote: Hi, On Fri, May 31, 2013 at 9:14 PM, Robert Munteanu romb...@apache.org wrote: ...It seems that the ErrorHandlingTest fails sporadically when run inside a full maven build... I fixed a related issue in SLING-2818 a few weeks

Re: Disabling flaky tests

2013-06-03 Thread Carsten Ziegeler
Hi, for the error handling tests we should fix SLING-2724 and then revisit the tests. I looked at them once and I think they used some wrong assumptions at that time. But I can't recall this right now. Anyway, SLING-2724 needs to be fixed anyway, I'll try to get this done this week Carsten

Re: Disabling flaky tests

2013-06-03 Thread Bertrand Delacretaz
Hi, On Mon, Jun 3, 2013 at 2:17 PM, Carsten Ziegeler cziege...@apache.org wrote: ...SLING-2724 needs to be fixed anyway, I'll try to get this done this week... Ok - note that the ErrorHandlingTest fails semi-randomly, so not sure if that's related. I've just added retries to those tests

Re: Disabling flaky tests

2013-06-02 Thread Eric Norman
Personally, I'm not a big fan of hiding flaky/failing tests since it tends to remove some of the motivation to stabilize/fix them in a timely manner. That's my 2 cents. Regards, Eric On Fri, May 31, 2013 at 12:14 PM, Robert Munteanu romb...@apache.orgwrote: Hi, It seems that the

Re: Disabling flaky tests

2013-06-02 Thread Felix Meschberger
I agree here: Disabling the test and having an issue keeps the build green but bears the danger of forgetting about it ... Regards Felix Am 02.06.2013 um 16:04 schrieb Eric Norman: Personally, I'm not a big fan of hiding flaky/failing tests since it tends to remove some of the motivation to

Disabling flaky tests

2013-05-31 Thread Robert Munteanu
Hi, It seems that the ErrorHandlingTest fails sporadically when run inside a full maven build. I've tried locating the root cause for a couple of hours but failed. For this test, and for future flaky/failing tests, I suggest that we 1. Create an issue for the failing test 2. Disable the test and