On Mon, Aug 14, 2017 at 10:21:13AM -0700, Dylan Baker wrote: > Quoting Arkadiusz Hiler (2017-08-14 05:09:01) > > Currently, if a test from provided testlist fails to be discovered by > > the framework, piglit blows up with an exception. > > > > This is both good - for consistency/early errors - and bad - for > > handling some CI/automation scenarios (e.g autobisecting the tests). > > > > So let's keep the current default, but allow some flexibility with the > > new option that reports the missing tests as 'notrun'. > > > > v2: > > - change switch name to 'ignore', as skip is too suggestive > > - use DummyTest to get 'notrun' result instead of warnings > > > > v3: don't use OPTIONS > > > > bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99649 > > Cc: Dylan Baker <[email protected]> > > Cc: Tomi Sarvela <[email protected]> > > Cc: Martin Peres <[email protected]> > > Signed-off-by: Arkadiusz Hiler <[email protected]> > > --- > > framework/profile.py | 10 +++++++--- > > framework/programs/run.py | 12 ++++++++++++ > > framework/test/base.py | 12 ++++++++++++ > > 3 files changed, 31 insertions(+), 3 deletions(-) > > > > diff --git a/framework/profile.py b/framework/profile.py > > index a625318..cf0e298 100644 <SNIP> > > @@ -314,7 +315,10 @@ class TestProfile(object): > > if self.forced_test_list: > > opts = collections.OrderedDict() > > for n in self.forced_test_list: > > - opts[n] = self.test_list[n] > > + if self.options['ignore_missing'] and n not in > > self.test_list: > > + opts[n] = DummyTest(name=n, result=status.NOTRUN) > > name and result are not keyword arguments, they're positional, so please write > it like this instead: > opts[n] = DummyTest(n, status.NOTRUN) > > As long as this passes the unit test suite with the above changed, > Reviewed-by: Dylan Baker <[email protected]>
% tox > with_patches % git checkout HEAD~ % tox > without_patches % diff -u <(grep -v PYTHONHASHSEED with_patches) <(grep -v PYTHONHASHSEED without_patches) | sprunge http://sprunge.us/VZDN Exactly identical pass rate and the same tests are failing. I guess that's a pass? :-) > Thanks for fixing that for me. Glad to see you back in action. v4 with your r-b is comming in a second and thanks for the review! -- Cheers, Arek _______________________________________________ Piglit mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/piglit
