>-----Original Message----- >From: Vladimir Ivanov [mailto:[EMAIL PROTECTED] >Sent: Friday, November 24, 2006 3:55 PM >To: [email protected] >Subject: Re: [testing] test exclude list: can't we have incremental >exclusions? > >On 11/24/06, Alexei Zakharov <[EMAIL PROTECTED]> wrote: >> >> Hi, >> >> > So we just should choice what is better: to break sometimes tests run >or >> to >> > forget enable test(s) on some platforms. >> >> Yesterday, when I was removing one of the beans tests from exclude >> lists, I feel a bit uncomfortable while updating >> exclude.linux.x86_64.xxx since I have no (easy) access to such systems >> and had no plans to run tests on it. IMHO (in the perfect world) the >> fact that I remove or add something from / to exclude.linux.x86_64 >> means I've at least ran tests for this platform and obtained some >> result. So let's have a common list, it's easier to deal with it >> psychologically. :-) > > >From my point of view, you should exclude test for x86 arch only and ask >somebody (through the dev list, for example) to verify it on x86_64.
I agree here. >As for above question: +1 for being optimists, i.e. to remove the test >> from common list if it passes on all platforms available to tester. +1. It's better to break build sometimes and face the problem ASAP rather than exclude a test forever :) And of course in the perfect world exclude lists will be empty for all modules. We should strive for it. Regards, Alexey. >OK. I want to be optimistic too :) >So let's describe it on wiki(?) with some instruction what should be done >in >case of test failure. > > Thanks, Vladimir > >Thanks, >> >> 24.11.06, Vladimir Ivanov<[EMAIL PROTECTED]> написал(а): >> > On 11/24/06, Ivanov, Alexey A <[EMAIL PROTECTED]> wrote: >> > >> > > Vladimir, all, >> > > >> > > These are good questions. On the other hand, there are reasons to >> consider >> > > most of the tests to be not platform dependent. >> > > >> > > If you fixed an issue, and it's in "all" exclude list, you would >> remove it >> > > from there and check on the platforms available to you. If something >> goes >> > > wrong on other platforms, those will report the problem back. Then >you >> > > either fix the problem again, or add this test to the >> platform-specific >> > > exclude list. >> > > >> > > If you find some test fails, it's better to alert the community about >> the >> > > issue and file a JIRA issue. After the evaluation of the problem, a >> decision >> > > will be worked out how to exclude it: for all platforms or only for >> one >> > > specific, if the problem can't be easily fixed. >> > > >> > > Does it make sense? >> > >> > >> > >> > So we just should choice what is better: to break sometimes tests run >or >> to >> > forget enable test(s) on some platforms. >> > My idea that is normal situation for ours exclude files is empty list. >> In >> > this case does not matter how many of them we have. But when we enable >> new >> > platform we use platform specific excludes. >> > >> > We may have any procedure to exclude/enable tests but it will nice if >it >> > will one procedure for all :) >> > >> > >> > >> > > I am against duplication of the lists because it may easily hide a >> problem >> > > on other platform because someone forgot to update all 8 lists, for >> example. >> > >> > >> > I don't like duplication too but I think it is 'initial' state only. In >> the >> > nearest feature (I hope) we enable tests at least for win/lnx on x86 :) >> > >> > Thanks, Vladimir >> >> -- >> Alexei Zakharov, >> Intel Enterprise Solutions Software Division >> -- Alexey A. Ivanov Intel Enterprise Solutions Software Division
