We have recently started using jenkins here at Intel to do regression testing across multiple generations of hardware. Because of this we have become concerned with the large number of tests that fail and the large number of tests that skip, specifically on older platforms.
This series is the first 15 patches shaved off of a larger series I'm working on to add better test skipping in the framework. I have targeted the framework as the best place to skip tests for a couple of reasons. First, it allows us to use metadata in the various profiles as skip criteria; in this series platforms (glx, egl, gbm, etc) are used to skip tests when that platform isn't available. In a later patches I'm working on we're able to use GL support as that criteria, allowing old platforms to skip GL features that were not part of that hardware's targeted GL API. Doing this work I took some time to cleanup the piglit Test code, I created a test package to hold the growing number of Test classes, split some code into Mixins, extended the early return code in Test.run() to be more versatile, split the PiglitTest class into PiglitGLTest and PiglitCLTest classes, pulled the code that adds -fbo and -auto into PiglitGLTest's command getter, and cleaned up some potential bugs. All of these changes result in better code, better code sharing between OpenGL and OpenCL tests (which means basically OpenCL doesn't run OpenGL specific code which only increases their runtime), and ultimately the ability to skip tests based on platform. This series implements fast skipping for two types of tests, glx tests on non glx, and egl tests on glx. I was somewhat unsure about the right way to skip the egl tests. Currently all platforms except pure glx support egl (x11_egl, mixed_glx_egl, and wayland always do), or can be configured with egl support (gbm). I chose to exclude glx, but I would be equally opened to include x11_egl, mixed_glx_egl, gbm, and wayland instead. The difference in runtime for a test that fast skips vs one that slow skips is dramatic. I measured that it takes about .00005 seconds for a test to fast skip, and about .1 seconds to slow skip. That adds up very quickly if your platform skips 20000 tests. _______________________________________________ Piglit mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/piglit
