On Wed, Mar 19, 2014 at 03:43:13PM +0000, Thomas Wood wrote: > The following two patches add support for a timeout status and then implement > a > timeout mechanism in igt.py. > > The "timeout" status is currently positioned just after "pass" in the status > ordering. This means that a timeout status does not introduce a regression > from > other failure states, since the actual test status is unknown, but a change > from timeout to fail is considered a regression. >
What is the rationale for making timeout->fail a regression? Based on the way I use piglit, I think it would be better to have the reverse (i.e fail->timeout is a regression), because in a real application it seems like hanging the system is much worse than mis-rendering something. Also, the normal work flow for fixing a hanging test is first fix the hang and then fix the test, so I think it makes sense to treat timeout->fail as a fix. -Tom > Thomas Wood (2): > framework: add support for a "timeout" status > tests/igt.py: stop tests after 10 minutes > > framework/log.py | 2 +- > framework/status.py | 20 +++++---- > framework/tests/status_tests.py | 4 +- > templates/index.css | 5 ++- > tests/igt.py | 89 > +++++++++++++++++++++++++++++++++++++++++ > 5 files changed, 108 insertions(+), 12 deletions(-) > > -- > 1.8.5.3 > > _______________________________________________ > Piglit mailing list > [email protected] > http://lists.freedesktop.org/mailman/listinfo/piglit _______________________________________________ Piglit mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/piglit
