Quoting Fabian Bieler (2018-01-20 05:38:45) > If a subtest crashed it and all following subtests have the result notrun. > Only the test as a whole has the result crash. > This commit propagates the crash to the correct subtest. > > Note that if the crash happened after the last subtest it is silently ignored. > --- > I like this approach. Personally, I would have added a NULL terminated array > of char* to config that defaults to NULL and emitted the JSON structure in > PIGLIT_GL_TEST_CONFIG_END, but that's just details.
I'm not a great C/C++ programmer, so if you have a better suggestion on how to enumerate the subtests I'd be glad to do that instead. > > Also this needs a little more work in the python framework: As is crashes > don't show up in the result summary (regardless of whether the crash happens > before or after the first call to piglit_report_subtest_result). > > If we use an OrderedDict instead of a regular dict in patch [3/9] and as the > container of Subtests (framework/results.py:44) we could deduce the crashing > subtest (if the order is preserved between piglit_enumerate_subtests and test > execution) with this patch. You're right. I had a versions of this I was working on at one point, but apparently lost when migrating machines or deleted :/ I think using an ordered dict would be better, in that case we can mark the correct subtest as crashed during the test run and not need to mess with any other code. > > Note that this would differ from the previous behavior in that if a test > crashes before a call to piglit_report_subtest_result (as is common in CL > tests, I am told) the first subtest is marked as crashed and the whole test > doesn't show up in the summary. > > > I'll gladly help with the adaption of the remaining tests if needed. > > framework/results.py | 18 ++++++++++++++++++ > 1 file changed, 18 insertions(+) > > diff --git a/framework/results.py b/framework/results.py > index 99dd3735b..2e1b832fa 100644 > --- a/framework/results.py > +++ b/framework/results.py > @@ -69,6 +69,22 @@ class Subtests(collections.MutableMapping): > res['__type__'] = 'Subtests' > return res > > + def set_crash(self): > + """Find the first subtest marked as notrun and mark it as crashed. > + > + If a subtest crashed it and all following subtests have the result > + notrun. Only the test as a whole has the result crash. > + This function propagates the crash to the correct subtest. > + Note that if the crash happened after the last subtest it is silently > + ignored. > + """ > + if status.CRASH in six.itervalues(self): > + return > + for key, result in six.iteritems(self): > + if result == status.NOTRUN: > + self[key] = status.CRASH > + break > + > @classmethod > def from_dict(cls, dict_): > if '__type__' in dict_: > @@ -232,6 +248,8 @@ class TestResult(object): > # Set special instances > if 'subtests' in dict_: > inst.subtests = Subtests.from_dict(dict_['subtests']) > + if inst.result == status.CRASH: > + inst.subtests.set_crash() > if 'time' in dict_: > inst.time = TimeAttribute.from_dict(dict_['time']) > > -- > 2.15.1 >
signature.asc
Description: signature
_______________________________________________ Piglit mailing list Piglit@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/piglit
