In my experience, many tests were written pre-test-verify, and don't clean
up correctly to deal with multiple runs in the same process. They work fine
when running as a single session, but blow up in TV. Having skip-if(verify)
means that we can at least mark those tests as known broken on TV without
taking them out of the test suites completely, especially if we find them
in places that we don't know how to fix them and need to file to someone
else who may not currently have time to fix.

On Tue, Mar 6, 2018 at 10:49 AM, Emilio Cobos Álvarez <emi...@crisal.io>
wrote:

> On 03/06/2018 06:04 PM, Geoffrey Brown wrote:
> > It is now possible to skip tests in test-verify. Simplify annotate the
> > manifest for your test:
> >
> > [test]
> > skip-if = verify
> >
> > or, for reftests:
> >
> > skip-if(verify) ...
> >
> > and the test-verify (TV) test task will not try to verify the annotated
> > test.
> >
> > Please don't abuse this feature! Most TV failures indicate a weakness in
> > the test.
>
> Could you point to an example of a "good" use of this feature? Is it
> just to avoid TV failing too intermittently? Is TV unable to run some
> tests? When does TV provide no value?
>
> Thanks!
>
>  -- Emilio
> _______________________________________________
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Reply via email to