Re: [classlib][testing] what is default behavior for intermittently failed tests exclude lists?
yes, but not by CC, and also, if we can find a way to tag them in the output so when we see a failure, and it's tagged as "intermittent", the developer knows to run it again rather than go hunting. I really do think that we'd be well served by fixing the intermittent tests - they point at something broken that we just don't understand, and that gives me the heebie-jeebies. I don't mind things that are known to be broken, if we understand why, but mysteries? no thanks :) geir On Mar 12, 2007, at 10:00 AM, Alexey Petrenko wrote: +1 for running intermittently failed tests by default 2007/3/12, Vladimir Ivanov <[EMAIL PROTECTED]>: Hello everyone, I want to discuss issue 3352 while it changes the current behavior of excluding policy for intermittently failed tests. I suggest running intermittently failed test by default. So the resulting exclude list by default will be combined as common + platform instead of common + platform + intermittent. Is it OK? thanks, Vladimir
Re: [classlib][testing] what is default behavior for intermittently failed tests exclude lists?
FYI I've applied Vladimir's patch for HARMONY-3352. So from now we will have all intermittent tests included by default. Note to all CC owners: please add "-Dexclude.interm=true" to your CC configs since we have agreed not to run intermttent tests on CC. Thanks, 2007/3/12, Alexey Petrenko <[EMAIL PROTECTED]>: +1 for running intermittently failed tests by default 2007/3/12, Vladimir Ivanov <[EMAIL PROTECTED]>: > Hello everyone, > I want to discuss issue 3352 while it changes the current behavior of > excluding policy for intermittently failed tests. I suggest running > intermittently failed test by default. So the resulting exclude list > by default will be combined as common + platform instead of common + > platform + intermittent. > Is it OK? > > thanks, Vladimir -- Alexei Zakharov, Intel ESSD
Re: [classlib][testing] what is default behavior for intermittently failed tests exclude lists?
+1 for running intermittently failed tests by default 2007/3/12, Vladimir Ivanov <[EMAIL PROTECTED]>: Hello everyone, I want to discuss issue 3352 while it changes the current behavior of excluding policy for intermittently failed tests. I suggest running intermittently failed test by default. So the resulting exclude list by default will be combined as common + platform instead of common + platform + intermittent. Is it OK? thanks, Vladimir
Re: [classlib][testing] what is default behavior for intermittently failed tests exclude lists?
Now I see. I missed "instead of" in Vladimir's message. Thanks! Mikhail 2007/3/12, Alexei Zakharov <[EMAIL PROTECTED]>: As far as I understand Vladimir's question was should we include intermittently failed test by default and exclude them only if a special switch is given or vice versa. I.e. about the build switch semantics. Personally I don't care. Just be prepared to possible failures and ready to run recently donated script :) Regards, 2007/3/12, Mikhail Loenko <[EMAIL PROTECTED]>: > IIRC we agreed to exclude intermittently failing tests from CC (to > avoid false alarms) > and to include them to pre-commit testing (to benmefit from other > valid tests that are excluded in the same bundle) > > Thanks, > Mikhail > > 2007/3/12, Vladimir Ivanov <[EMAIL PROTECTED]>: > > Hello everyone, > > I want to discuss issue 3352 while it changes the current behavior of > > excluding policy for intermittently failed tests. I suggest running > > intermittently failed test by default. So the resulting exclude list > > by default will be combined as common + platform instead of common + > > platform + intermittent. > > Is it OK? > > > > thanks, Vladimir > > > -- Alexei Zakharov, Intel ESSD
Re: [classlib][testing] what is default behavior for intermittently failed tests exclude lists?
As far as I understand Vladimir's question was should we include intermittently failed test by default and exclude them only if a special switch is given or vice versa. I.e. about the build switch semantics. Personally I don't care. Just be prepared to possible failures and ready to run recently donated script :) Regards, 2007/3/12, Mikhail Loenko <[EMAIL PROTECTED]>: IIRC we agreed to exclude intermittently failing tests from CC (to avoid false alarms) and to include them to pre-commit testing (to benmefit from other valid tests that are excluded in the same bundle) Thanks, Mikhail 2007/3/12, Vladimir Ivanov <[EMAIL PROTECTED]>: > Hello everyone, > I want to discuss issue 3352 while it changes the current behavior of > excluding policy for intermittently failed tests. I suggest running > intermittently failed test by default. So the resulting exclude list > by default will be combined as common + platform instead of common + > platform + intermittent. > Is it OK? > > thanks, Vladimir > -- Alexei Zakharov, Intel ESSD
Re: [classlib][testing] what is default behavior for intermittently failed tests exclude lists?
2007/3/12, Alexei Zakharov <[EMAIL PROTECTED]>: IMO to make use of this we probably need a script that generates combined list of known intermittent failures for all modules. So the committer can quickly check if the observed failure is Ok (i.e. the test by itself fails from time to time) or not. cat modules/*/make/*interm ASF license granted :) Thanks, Mikhail Regards, 2007/3/12, Vladimir Ivanov <[EMAIL PROTECTED]>: > Hello everyone, > I want to discuss issue 3352 while it changes the current behavior of > excluding policy for intermittently failed tests. I suggest running > intermittently failed test by default. So the resulting exclude list > by default will be combined as common + platform instead of common + > platform + intermittent. > Is it OK? > > thanks, Vladimir -- Alexei Zakharov, Intel ESSD
Re: [classlib][testing] what is default behavior for intermittently failed tests exclude lists?
IIRC we agreed to exclude intermittently failing tests from CC (to avoid false alarms) and to include them to pre-commit testing (to benmefit from other valid tests that are excluded in the same bundle) Thanks, Mikhail 2007/3/12, Vladimir Ivanov <[EMAIL PROTECTED]>: Hello everyone, I want to discuss issue 3352 while it changes the current behavior of excluding policy for intermittently failed tests. I suggest running intermittently failed test by default. So the resulting exclude list by default will be combined as common + platform instead of common + platform + intermittent. Is it OK? thanks, Vladimir
Re: [classlib][testing] what is default behavior for intermittently failed tests exclude lists?
IMO to make use of this we probably need a script that generates combined list of known intermittent failures for all modules. So the committer can quickly check if the observed failure is Ok (i.e. the test by itself fails from time to time) or not. Regards, 2007/3/12, Vladimir Ivanov <[EMAIL PROTECTED]>: Hello everyone, I want to discuss issue 3352 while it changes the current behavior of excluding policy for intermittently failed tests. I suggest running intermittently failed test by default. So the resulting exclude list by default will be combined as common + platform instead of common + platform + intermittent. Is it OK? thanks, Vladimir -- Alexei Zakharov, Intel ESSD
[classlib][testing] what is default behavior for intermittently failed tests exclude lists?
Hello everyone, I want to discuss issue 3352 while it changes the current behavior of excluding policy for intermittently failed tests. I suggest running intermittently failed test by default. So the resulting exclude list by default will be combined as common + platform instead of common + platform + intermittent. Is it OK? thanks, Vladimir
