On Mon, Jul 20, 2015 at 01:57:11PM +0100, Thomas Wood wrote: > On 13 July 2015 at 23:09, Dylan Baker <[email protected]> wrote: > > > > On Mon, Jul 13, 2015 at 02:09:26PM +0100, Thomas Wood wrote: > > > On 10 July 2015 at 19:24, Dylan Baker <[email protected]> wrote: > > > > > > > Hi Thomas, > > > > > > > > I meant to get back to you earlier on this, but I forgot yesterday when > > > > I got bogged down in Jenkins madness. > > > > > > > > Anyway, if you have a look at the (I think) 2nd patch in my python3 > > > > series, you'll notice that I had the idea that we could set timeouts in > > > > classes on a per class basis. This presents a problem for using a > > > > universal test time overwrite, especially when you consider that we also > > > > may need per-test overwrites (some of the piglit tests are really long, > > > > some of them are really short). > > > > > > > > Do you want to have an igt only override, or should we try to figure out > > > > how to make something that will work universally? > > > > > > > > > > The idea was to allow the user to temporarily change the timeout that > > > applied to all tests. It could include a parameter to ensure that timeout > > > durations are only ever reduced or increased. > > > > > > > Okay, well here's the real problem I see. I designed the timeouts such > > that the class sets a sane timeout for the test class in general, and > > then tests that fall outside that norm can override. So the > > implementation looks something like: > > > > ```python > > class BaseTest(object): > > timeout = 50 > > > > class Test(BaseTest): > > timeout = 100 > > > > class Test2(BaseTest): > > pass > > > > foo = Test2() > > foo.timeout = 400 > > ``` > > > > Now the problem with this plus your model is that either: > > we override the BaseTest class timeout then any test the defines a > > timeout parameter (like Test) will not receive that override from > > BaseTest, nor will the foo instance, since it sets its own timeout. > > > > Could the subclasses check the timeout specified by the user and > decide whether or not to apply it?
What about just adding a --max-timeout and --min-timeout for the cmdline? I think that should cover all uses-cases we have in mind really. Then we could use that to adjust the per-testcaes timeout in a suitable way. -Daniel > > > > > > > > In python3 I think we can solve this via a class decorator, but for > > python2 that's a much harder proposition to fix. Although maybe it > > doesn't matter as much in python2, since timeouts are not thread safe, > > not portable, and are only implemented by igt in python2 > > > > But I'd really like to keep timeouts a property of each test class, > > because reasonable timeout durations really are not universal. 120 > > seconds is pretty reasonable for glean, 30 for most piglit tests, 10 for > > glsl parser tests, and 600 for igt tests, but we don't want to waith 600 > > seconds for a glsl parser test to return, if it doesn't return after 10 > > seconds something is wrong. > > > > Dylan > > > > [snip] > _______________________________________________ > Piglit mailing list > [email protected] > http://lists.freedesktop.org/mailman/listinfo/piglit -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ Piglit mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/piglit
