This allows shader_runner to consume more than one test at a time. When it does this it reports the results as subtests rather than as regular tests. It also updates the python framework to handle and resume from crashes in the tests. When this mode is enabled, shader_runner runs an entire directory of shader_test files at a time.
This makes shader_runner (which takes up the bulk of piglit) run much faster, even compared to per-process with fast skipping enabled, but at the cost of the robustness and reproducibility of process isolation. For that reason its an option that must be enabled, the default should always be robustness not speed. This can be turned on permanently on in piglit.conf. >From a performance standpoint it's impressive (this is a 4 thread HSW): PIGLIT_NO_FAST_SKIP=1 ./piglit run shader ns_w_is -c 936.35s user 611.18s system 301% cpu 8:33.71 total ./piglit run shader fs_w_is -c 726.21s user 398.25s system 306% cpu 6:07.23 total PIGLIT_NO_FAST_SKIP=1 ./piglit run shader ns_no_is -c --process-isolation=false 334.21s user 122.04s system 201% cpu 3:46.55 total ./piglit run shader fs_no_is -c --process-isolation=false 160.47s user 7.83s system 293% cpu 57.327 total For the general use case (With fast skipping) this is a reduction of 5 minutes of runtime. This work is part mine, and partially from Marek. Changes since v1: - Add the second squash! commit, which is all v1 changes The biggest changes are fixing more GL vs GLES differences, doing all of our guarding at the front of the function, and doing a better job cleaning up between tests. Dylan Baker (10): framework/results: Lower subtest names framework: Split the file parsing code out of ShaderTest framework: Split FastSkipMixin. framework: Bump JSON on disk format to version 9 framework: Add a Mixin class for running multiple tests in a single process squash! shader_runner: add ability to run multiple tests per process squash! shader_runner: add ability to run multiple tests per process framework: Add class for running multiple shader_tests in a single process framework: Plug in fast-skipping to MultiShaderTest framework: add boilerplate to turn off process isolation Marek Ol????k (6): shader_runner: propagate all errors to piglit_init util: guard against double context destruction util: always set piglit_is_core_profile util: rename piglit_gl_reinitialize_extensions->piglit_gl_invalidate_extensions util: move PIGLIT_PATH_SEP to piglit-util.h shader_runner: add ability to run multiple tests per process framework/backends/json.py | 18 +- framework/options.py | 1 +- framework/programs/run.py | 23 +- framework/results.py | 8 +- framework/test/base.py | 155 ++- framework/test/deqp.py | 4 +- framework/test/opengl.py | 182 ++- framework/test/shader_test.py | 201 ++- piglit.conf.example | 7 +- tests/all.py | 37 +- tests/shader.py | 4 +- tests/shaders/shader_runner.c | 582 +++++++--- tests/util/piglit-framework-gl.c | 4 +- tests/util/piglit-framework-gl/piglit_glut_framework.c | 7 +- tests/util/piglit-framework-gl/piglit_wfl_framework.c | 5 +- tests/util/piglit-util-gl.c | 2 +- tests/util/piglit-util-gl.h | 2 +- tests/util/piglit-util.c | 6 +- tests/util/piglit-util.h | 6 +- unittests/framework/backends/schema/piglit-9.json | 122 ++- unittests/framework/backends/shared.py | 4 +- unittests/framework/backends/test_json_update.py | 85 +- unittests/framework/test/test_base.py | 138 +- unittests/framework/test/test_opengl.py | 192 +-- unittests/framework/test/test_shader_test.py | 63 +- 25 files changed, 1533 insertions(+), 325 deletions(-) create mode 100644 unittests/framework/backends/schema/piglit-9.json -- git-series 0.8.10
_______________________________________________ Piglit mailing list Piglit@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/piglit