On Wed, Mar 19, 2014 at 09:15:17AM -0700, Tom Stellard wrote: > On Wed, Mar 19, 2014 at 03:43:13PM +0000, Thomas Wood wrote: > > The following two patches add support for a timeout status and then > > implement a > > timeout mechanism in igt.py. > > > > The "timeout" status is currently positioned just after "pass" in the status > > ordering. This means that a timeout status does not introduce a regression > > from > > other failure states, since the actual test status is unknown, but a change > > from timeout to fail is considered a regression. > > > > What is the rationale for making timeout->fail a regression? Based on > the way I use piglit, I think it would be better to have the reverse (i.e > fail->timeout is a regression), because in a real application it seems > like hanging the system is much worse than mis-rendering something. Also, > the normal work flow for fixing a hanging test is first fix the hang > and then fix the test, so I think it makes sense to treat timeout->fail > as a fix.
Well in igt timeout is somewhat a normal condition on some machines because on badly underpowered platforms some of the stress-tests just take forever. We've tried to tune ofc but there's still tests that take forever. I think you make a good case that in general timeout should be treated more severe than fail, so I'm ok if we switch this around. And we also have that problem ofc when the driver hangs somewhere. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ Piglit mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/piglit
