On 03/06/2018 08:05 PM, Kyle Machulis wrote: > In my experience, many tests were written pre-test-verify, and don't clean > up correctly to deal with multiple runs in the same process. They work fine > when running as a single session, but blow up in TV. Having skip-if(verify) > means that we can at least mark those tests as known broken on TV without > taking them out of the test suites completely, especially if we find them > in places that we don't know how to fix them and need to file to someone > else who may not currently have time to fix.
Ahá, that makes perfect sense, thanks a lot! -- Emilio > On Tue, Mar 6, 2018 at 10:49 AM, Emilio Cobos Álvarez <emi...@crisal.io> > wrote: > >> On 03/06/2018 06:04 PM, Geoffrey Brown wrote: >>> It is now possible to skip tests in test-verify. Simplify annotate the >>> manifest for your test: >>> >>> [test] >>> skip-if = verify >>> >>> or, for reftests: >>> >>> skip-if(verify) ... >>> >>> and the test-verify (TV) test task will not try to verify the annotated >>> test. >>> >>> Please don't abuse this feature! Most TV failures indicate a weakness in >>> the test. >> >> Could you point to an example of a "good" use of this feature? Is it >> just to avoid TV failing too intermittently? Is TV unable to run some >> tests? When does TV provide no value? >> >> Thanks! >> >> -- Emilio >> _______________________________________________ >> dev-platform mailing list >> dev-platform@lists.mozilla.org >> https://lists.mozilla.org/listinfo/dev-platform >> > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
