On Sat, Sep 10, 2016, at 06:56, Marek Olšák wrote: > Thanks a lot of doing this!! > > BTW, --no-process-isolation is not a valid parameter.
Yeah, I decided to change the parameter after I ran the benchmarks, when I was writing documentation and proper commit messages. I realized that I'd promised a piglit.conf option to force this on, and that would either require two switches, or just changing the switch to take a parameter to get consistent behavior. So instead it's '--process-isolation' with an argument that can be roughly translated as True or False (1, 0, true, false, yes, no; with any capitalization). I forgot to update the cover letter. Dylan > > Marek > > On Fri, Sep 9, 2016 at 9:18 PM, Dylan Baker <dy...@pnwbakers.com> wrote: > > This allows shader_runner to consume more than one test at a time. When > > it does this it reports the results as subtests rather than as regular > > tests. It also updates the python framework to handle and resume from > > crashes in the tests. When this mode is enabled, shader_runner runs an > > entire directory of shader_test files at a time. > > > > This makes shader_runner (which takes up the bulk of piglit) run much > > faster, even compared to per-process with fast skipping enabled, but at > > the cost of the robustness and precision of process isolation. For that > > reason its an option that must be enabled, the default should always be > > robustness not speed. This can be forced on in piglit.conf. > > > > From a performance standpoint it's impressive (this is a 4 thread HSW): > > PIGLIT_NO_FAST_SKIP=1 ./piglit run shader ns_w_is -c > > 936.35s user 611.18s system 301% cpu 8:33.71 total > > ./piglit run shader fs_w_is -c > > 726.21s user 398.25s system 306% cpu 6:07.23 total > > PIGLIT_NO_FAST_SKIP=1 ./piglit run shader ns_no_is -c --no-process-isolation > > 334.21s user 122.04s system 201% cpu 3:46.55 total > > ./piglit run shader fs_no_is -c --no-process-isolation > > 160.47s user 7.83s system 293% cpu 57.327 total > > > > For the general use case (With fast skipping) this is a reduction of 5 > > minutes of runtime. With fast-skipping seems pretty useful, as it > > reduces runtime by 3 minutes even with the removal of process isolation. > > > > Dylan Baker (9): > > framework/results: Lower subtest names > > framework: Split the file parsing code out of ShaderTest > > framework: Split FastSkipMixin. > > framework: Bump JSON on disk format to version 9 > > framework: Add a Mixin class for running multiple tests in a single > > process > > squash! shader_runner: add ability to run multiple tests per process > > framework: Add class for running multiple shader_tests in a single process > > framework: Plug in fast-skipping to MultiShaderTest > > framework: add boilerplate to turn off process isolation > > > > Marek Olšák (6): > > shader_runner: propagate all errors to piglit_init > > util: guard against double context destruction > > util: always set piglit_is_core_profile > > util: rename > > piglit_gl_reinitialize_extensions->piglit_gl_invalidate_extensions > > util: move PIGLIT_PATH_SEP to piglit-util.h > > shader_runner: add ability to run multiple tests per process > > > > framework/backends/json.py | 18 +- > > framework/options.py | 1 +- > > framework/programs/run.py | 22 +- > > framework/results.py | 8 +- > > framework/test/base.py | 155 ++- > > framework/test/deqp.py | 4 +- > > framework/test/opengl.py | 182 ++- > > framework/test/shader_test.py | 201 +++- > > piglit.conf.example | 7 +- > > tests/all.py | 34 +- > > tests/shader.py | 4 +- > > tests/shaders/shader_runner.c | 560 +++++++--- > > tests/util/piglit-framework-gl.c | 4 +- > > tests/util/piglit-framework-gl/piglit_glut_framework.c | 7 +- > > tests/util/piglit-framework-gl/piglit_wfl_framework.c | 5 +- > > tests/util/piglit-util-gl.c | 2 +- > > tests/util/piglit-util-gl.h | 2 +- > > tests/util/piglit-util.c | 6 +- > > tests/util/piglit-util.h | 6 +- > > unittests/framework/backends/schema/piglit-9.json | 122 ++- > > unittests/framework/backends/shared.py | 4 +- > > unittests/framework/backends/test_json_update.py | 85 ++- > > unittests/framework/test/test_base.py | 138 +- > > unittests/framework/test/test_opengl.py | 192 +-- > > unittests/framework/test/test_shader_test.py | 63 +- > > 25 files changed, 1514 insertions(+), 318 deletions(-) > > create mode 100644 unittests/framework/backends/schema/piglit-9.json > > > > -- > > git-series 0.8.10 -- Dylan Baker dy...@pnwbakers.com _______________________________________________ Piglit mailing list Piglit@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/piglit