[Piglit] [PATCH 08/15] tests/fbo-storage-formats: Always print the same number of subtests

!skip -> skip won't show up in the regressions/fixes/etc lists anyway, and this means that the output will always be the same. --- tests/fbo/fbo-storage-formats.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/tests/fbo/fbo-storage-formats.c b/tests/fbo/fbo-storage-formats.c index

[Piglit] [PATCH 12/15] tests: always report subtest results for degenerate-prims

--- tests/general/degenerate-prims.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/tests/general/degenerate-prims.c b/tests/general/degenerate-prims.c index 7ed6093..d2bb7d6 100644 --- a/tests/general/degenerate-prims.c +++ b/tests/general/degenerate-prims.c @@ -63,10

[Piglit] [PATCH 15/15] Add series todo

--- todo | 108 - 1 file changed, 108 insertions(+) create mode 100644 todo diff --git a/todo b/todo new file mode 100644 index 000..c869582 --- /dev/null +++ b/todo @@ -0,0 +1,108 @@ +tests/cl/api/set-kernel-arg.c

[Piglit] [PATCH 01/15] tests/util: Add config value to enumerate subtests

This adds a new member to the GL config struct for informing the python framework the number and order of subtests that will be run (if any). If this value is unset then no subtests are expected. --- tests/util/piglit-framework-gl.c | 13 + tests/util/piglit-framework-gl.h | 11

[Piglit] [PATCH 06/15] tests: enumerate subtests in gl-2.1-gbo test

This gives us something to test the python part against. Signed-off-by: Dylan Baker --- tests/spec/gl-2.1/pbo.c | 12 1 file changed, 12 insertions(+) diff --git a/tests/spec/gl-2.1/pbo.c b/tests/spec/gl-2.1/pbo.c index 83dc1c4..81d4f64 100644 ---

[Piglit] [PATCH 13/15] tests: enumerate subtests in degenerate-prims

--- tests/general/degenerate-prims.c | 59 ++--- 1 file changed, 41 insertions(+), 18 deletions(-) diff --git a/tests/general/degenerate-prims.c b/tests/general/degenerate-prims.c index d2bb7d6..6a73b65 100644 --- a/tests/general/degenerate-prims.c +++

[Piglit] [PATCH 07/15] tests: enumerate subtests in fbo-incomplete

Signed-off-by: Dylan Baker --- tests/fbo/fbo-incomplete.cpp | 12 1 file changed, 12 insertions(+) diff --git a/tests/fbo/fbo-incomplete.cpp b/tests/fbo/fbo-incomplete.cpp index 8cde6d2..8d16ec1 100644 --- a/tests/fbo/fbo-incomplete.cpp +++

[Piglit] [PATCH 05/15] framework: if a test with subtests crashes mark the offending subtest

This relies on the fact that subtests are guaranteed to be ordered to mark the crashing subtest as such. This ensures that the correct status will be propagated up the totals tree. --- framework/test/base.py| 8 ++- unittests/framework/test/test_base.py | 36

[Piglit] [PATCH 03/15] framework: use statuses instead of relying on string conversions

This is a small cleanup, but we need to touch this code further for subtest enumeration work, and we'll want to use statuses there too. --- framework/test/base.py | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/framework/test/base.py b/framework/test/base.py index

[Piglit] [PATCH 09/15] tests/fbo-storage-formats: Make subtest names predictable

Currently the name of the subtest relies on checking the GL state of the command to know whether the framebuffer is completely or incomplete. That is a problem for enumerating subtests, we would need to know ahead of time whether the framebuffer is complete or not. Instead print the completeness

[Piglit] [PATCH 11/15] tests/fbo-storage-formats: enumerate subtests

--- tests/fbo/fbo-storage-formats.c | 41 ++ 1 file changed, 27 insertions(+), 14 deletions(-) diff --git a/tests/fbo/fbo-storage-formats.c b/tests/fbo/fbo-storage-formats.c index 7f13684..ee22883 100644 --- a/tests/fbo/fbo-storage-formats.c +++

[Piglit] [PATCH 10/15] tests/fbo-storage-formats: print subtest result for skip too

Because we always want to have the same number of subtests printed, or they'll show up as "NOTRUN" instead of "SKIP" in the summary. Signed-off-by: Dylan Baker --- tests/fbo/fbo-storage-formats.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git

[Piglit] [PATCH 14/15] tests: enumerate subtests in linestipple

--- tests/general/linestipple.c | 24 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/tests/general/linestipple.c b/tests/general/linestipple.c index ba0924e..6626b3c 100644 --- a/tests/general/linestipple.c +++ b/tests/general/linestipple.c @@ -28,14 +28,6

[Piglit] [PATCH 02/15] framework: use an OrderedDict for subtest storage

This makes the order of the subtests deterministic. Since piglit test binaries are expected to enumerate subtests in the same order that they'll run them this will allow us to know which subtest crashed when there is a crash. --- framework/results.py | 2 +- 1 file changed, 1 insertion(+), 1

[Piglit] [PATCH 04/15] framework: add support for parsing subtest enumeration

This adds support for enumerating subtests to the python layer. When it sees this it sets each subtest to notrun. This allows the python framework to report that tests didn't run when they were expected to. Signed-off-by: Dylan Baker --- framework/test/piglit_test.py |

Re: [Piglit] Question regarding Crucible TODO item

On Thu, Jan 18, 2018 at 11:45 AM, Fabian Bieler wrote: > Hello! > > The Crucible TODO.txt file contains: > - Add a --diff option to crucible-run that dumps image diffs for failed > tests. > (Requires multiple command buffers). > > I do not understand how an optional

Re: [Piglit] [PATCH] framework: Propagate crash to subtest.

Quoting Fabian Bieler (2018-01-20 05:38:45) > If a subtest crashed it and all following subtests have the result notrun. > Only the test as a whole has the result crash. > This commit propagates the crash to the correct subtest. > > Note that if the crash happened after the last subtest it is

Re: [Piglit] [PATCH v2] travis: add docker based cmake build job

On Tue, 2018-01-16 at 18:13 +0200, Andres Gomez wrote: > Until now we were only running the python unit tests. > > It seems desirable to also check that the CMake based build compiles > successfully. We do that now using docker. > > The docker build can be tweaked with some environment variables