On Wed, Nov 25, 2015 at 11:32:48AM -0500, Rob Clark wrote: > Complete hack, but maybe we want to make something like this an optional > way that piglit-summary dumps out results. > > Basically it groups all the results according to transition (ie. 'pass > -> fail' or 'fail -> fail -> pass', etc., and then for each group dumps > out the test environment and cmdline. > > This gives me something I can easily cut/paste to rerun. For example, > a common use-case for me while debugging some fix/feature/etc on the > driver side, is to diff results between a baseline run and most recent > run. And as I debug/fix regressions on driver side, I tend to first > want to re-run the set of tests that had gone pass->fail. The old way > involved './piglit-summary.py -d baseline latest | grep "pass fail"' > then finding the results.json (which is slightly more annoying because > of the whole s/@/\// thing) and cut/paste the cmdline. A somewhat time > consuming and annoying way to do things.
Just FYI, you dont ned to worry about the 's!/!@!g' thing, -t and -x handle that automagically ;) > > There is still the slight problem of how to escape special chars in > piglit cmdline. Seriously, cmdline args like "*Lod" are a horrible > idea. > --- > fwiw, example output: http://hastebin.com/raw/pezotoyoje > > framework/results.py | 11 +++++++++++ > framework/summary/console_.py | 33 ++++++++++++++++++++++++++++++--- > 2 files changed, 41 insertions(+), 3 deletions(-) > > diff --git a/framework/results.py b/framework/results.py > index eeffcb7..fa43cf6 100644 > --- a/framework/results.py > +++ b/framework/results.py > @@ -308,6 +308,17 @@ class TestrunResult(object): > except KeyError: > raise e > > + def get_result_object(self, key): > + """Similar to get_result() but returns result object""" > + try: > + return self.tests[key] > + except KeyError as e: > + name, test = grouptools.splitname(key) > + try: > + return self.tests[name] > + except KeyError: > + raise e > + > def calculate_group_totals(self): > """Calculate the number of pases, fails, etc at each level.""" > for name, result in self.tests.iteritems(): > diff --git a/framework/summary/console_.py b/framework/summary/console_.py > index d219498..ebc7adb 100644 > --- a/framework/summary/console_.py > +++ b/framework/summary/console_.py > @@ -89,10 +89,37 @@ def _print_summary(results): > > def _print_result(results, list_): > """Takes a list of test names to print and prints the name and result.""" > + # Setup a hashtable mapping transition (ie. 'pass -> fail' to list of > + # result objects for test that followed that transition (ie. first > + # result is 'pass' and second result is 'fail'). This could of course > + # mean transition strings like 'pass -> fail -> pass' if there were > + # three sets of results. I guess the normal use case would be to > + # compare two sets of results. > + # > + # Note that we just keep the last result object, but it is expected > + # that command/environment are the same across piglit runs. > + groups = {} > for test in sorted(list_): > - print("{test}: {statuses}".format( > - test=grouptools.format(test), > - statuses=' '.join(str(r) for r in results.get_result(test)))) > + transition = None > + last = None > + for each in results.results: > + status = str(each.get_result(test)) > + if transition is None: > + transition = status > + else: > + transition = ' -> '.join([transition, status]) > + last = each > + result = last.get_result_object(test) > + if not transition in groups: > + groups[transition] = [] > + groups[transition].append(result) > + > + # And now print out results grouped by transition. > + for transition, resultlist in groups.iteritems(): > + print(transition + ':') > + for result in resultlist: > + print(result.environment + ' ' + result.command) > + print('') > > > def console(results, mode): > -- > 2.5.0 > > _______________________________________________ > Piglit mailing list > Piglit@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/piglit Hi Rob, Somewhere I have a patch that automates this, you point it at two results and then give it a set of statuses and it reruns the tests that match. Something like: piglit rerurn res1 res2 new pass fail,warn,crash And it gives you a new result with just those changes. Is that what you're looking for? Dylan
signature.asc
Description: PGP signature
_______________________________________________ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
