Re: skip-if(verify)

2018-03-07 Thread Gabriele Svelto
I've also seen TV failures caused by flaws in the test harness which weren't visible before. See bug 1410165 [1] for example, it was caused by an observer being unregistered after the first iteration of a test had finished, causing the following iterations to fail. Gabriele [1] Permaorange

Re: skip-if(verify)

2018-03-06 Thread Emilio Cobos Álvarez
On 03/06/2018 08:05 PM, Kyle Machulis wrote: > In my experience, many tests were written pre-test-verify, and don't clean > up correctly to deal with multiple runs in the same process. They work fine > when running as a single session, but blow up in TV. Having skip-if(verify) > means that we can

Re: skip-if(verify)

2018-03-06 Thread Kyle Machulis
In my experience, many tests were written pre-test-verify, and don't clean up correctly to deal with multiple runs in the same process. They work fine when running as a single session, but blow up in TV. Having skip-if(verify) means that we can at least mark those tests as known broken on TV

Re: skip-if(verify)

2018-03-06 Thread Emilio Cobos Álvarez
On 03/06/2018 06:04 PM, Geoffrey Brown wrote: > It is now possible to skip tests in test-verify. Simplify annotate the > manifest for your test: > > [test] > skip-if = verify > > or, for reftests: > > skip-if(verify) ... > > and the test-verify (TV) test task will not try to verify the