Re: [Piglit] [PATCH 00/35] Serialize profiles into XML at build time

Quoting Juan A. Suarez Romero (2018-05-02 09:49:08) > Hi, Dylan. > > I see you've pushed this series. > > Now, when I'm trying to run some profiles (mainly, tests/crucible and > tests/khr_gl* ), seems they are broken: > > [/7776] > Traceback (most recent call last): > File "./piglit",

Re: [Piglit] [PATCH 1/5] framework/html: guard against errors writing individual test results

Quoting Marek Olšák (2018-05-02 13:32:43) > From: Nicolai Hähnle > > --- > framework/summary/html_.py | 18 +++--- > 1 file changed, 11 insertions(+), 7 deletions(-) > > diff --git a/framework/summary/html_.py b/framework/summary/html_.py > index

Re: [Piglit] [PATCH 4/5] khr_gl45: add support for mustpass lists

from khr_gl to khr_gl45. with that, Reviewed-by: Dylan Baker <dy...@pnwbakers.com> > > """ > > from __future__ import ( > absolute_import, division, print_function, unicode_literals > ) > import itertools > > from framework.test impo

Re: [Piglit] [PATCH 5/5] deqp_egl: add support for mustpass lists

Reviewed-by: Dylan Baker <dy...@pnwbakers.com> Quoting Marek Olšák (2018-05-02 13:32:47) > From: Nicolai Hähnle <nicolai.haeh...@amd.com> > > --- > tests/deqp_egl.py | 10 +++--- > 1 file changed, 7 insertions(+), 3 deletions(-) > > diff --git a/test

Re: [Piglit] [PATCH 3/5] framework: handle UnicodeDecodeError

Quoting Marek Olšák (2018-05-02 13:32:45) > From: Marek Olšák > > This happens due to LLVM printing colored text into stdout/stderr on error. > --- > framework/test/base.py | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/framework/test/base.py

Re: [Piglit] [PATCH 2/5] framework: fix grouptools.commonprefix when len(args) == 1

Reviewed-by: Dylan Baker <dy...@pnwbakers.com> Quoting Marek Olšák (2018-05-02 13:32:44) > From: Nicolai Hähnle <nicolai.haeh...@amd.com> > > --- > framework/grouptools.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/framework/groupt

[Piglit] [PATCH] find_static_tests.py: fix python2 compatibility

Because python2 uses bytes, but python3 uses unicode. CC: Michel Dänzer Fixes: d42d909cd754d0e2c41eec60f3a1015f2d882b95 ("tests: Add script to find all hand written test files") --- tests/find_static_tests.py | 11 ++- 1 file changed, 10 insertions(+), 1

Re: [Piglit] [PATCH 00/35] Serialize profiles into XML at build time

Quoting Rafael Antognolli (2018-04-25 11:01:29) > On Mon, Apr 23, 2018 at 10:47:08AM -0700, Dylan Baker wrote: > > I'm planning to just push this Wednesday if no one expresses any more > > concerns, > > or signals that they want time to test or review this. > > I

Re: [Piglit] [PATCH 00/35] Serialize profiles into XML at build time

I'm planning to just push this Wednesday if no one expresses any more concerns, or signals that they want time to test or review this. Dylan Quoting Dylan Baker (2018-04-04 15:26:48) > I don't expect everyone I've CC'd to give thorough review (or any > review), I've mostly CC'd people

Re: [Piglit] [PATCH] unittests: mustpass is a text list

ir.join('foo.xml') > -p.write(self._xml) > +p = tmpdir.join('foo.txt') > +p.write(self._txt) > tests = set(deqp.gen_mustpass_tests(six.text_type(p))) > +print(tests) > assert tests == { > 'dEQP.piglit.group1.test1', > 'dEQP.p

Re: [Piglit] [PATCH 00/35] Serialize profiles into XML at build time

Quoting Dylan Baker (2018-04-04 15:26:48) > I don't expect everyone I've CC'd to give thorough review (or any > review), I've mostly CC'd people who I think would be interested in this > work, or who's work flow I might be altered by it. > > Piglit has struggled to cope with the

Re: [Piglit] [PATCH] framework: update deqp mustpass list for text based mustpass

Quoting Dylan Baker (2018-04-11 13:39:26) > Quoting Eric Anholt (2018-04-11 12:09:34) > > Dylan Baker <dy...@pnwbakers.com> writes: > > > > > The current code has bit-rotted quite a bit, in late 2016 the format was > > > changed from XML based to tx

Re: [Piglit] [PATCH] framework: update deqp mustpass list for text based mustpass

Quoting Dylan Baker (2018-04-11 13:39:26) > Quoting Eric Anholt (2018-04-11 12:09:34) > > Dylan Baker <dy...@pnwbakers.com> writes: > > > > > The current code has bit-rotted quite a bit, in late 2016 the format was > > > changed from XML based to tx

Re: [Piglit] [PATCH] framework: update deqp mustpass list for text based mustpass

Quoting Eric Anholt (2018-04-11 12:09:34) > Dylan Baker <dy...@pnwbakers.com> writes: > > > The current code has bit-rotted quite a bit, in late 2016 the format was > > changed from XML based to txt based, but the code was never updated. > > --- > > I think y

[Piglit] [PATCH] framework: update deqp mustpass list for text based mustpass

The current code has bit-rotted quite a bit, in late 2016 the format was changed from XML based to txt based, but the code was never updated. --- framework/test/deqp.py | 22 ++ piglit.conf.example| 6 +++--- 2 files changed, 9 insertions(+), 19 deletions(-) diff --git

Re: [Piglit] [PATCH 00/35] Serialize profiles into XML at build time

Quoting Mark Janes (2018-04-10 18:54:46) > Dylan Baker <dy...@pnwbakers.com> writes: > > > Quoting Marek Olšák (2018-04-10 14:22:10) > >> On Tue, Apr 10, 2018 at 2:15 PM, Dylan Baker <dy...@pnwbakers.com> wrote: > >> > >> Quoting Eric Anhol

Re: [Piglit] [PATCH 00/35] Serialize profiles into XML at build time

Quoting Marek Olšák (2018-04-10 14:22:10) > On Tue, Apr 10, 2018 at 2:15 PM, Dylan Baker <dy...@pnwbakers.com> wrote: > > Quoting Eric Anholt (2018-04-09 17:10:35) > > Marek Olšák <mar...@gmail.com> writes: > > > > > Is this use case affec

Re: [Piglit] [PATCH 00/35] Serialize profiles into XML at build time

Quoting Eric Anholt (2018-04-09 17:10:35) > Marek Olšák writes: > > > Is this use case affected? > > > > piglit run --deqp-mustpass-list --process-isolation 0 -p gbm -c quick > > cts_gl45 deqp_gles2 deqp_gles3 deqp_gles31 > > > > Yes, that is just 1 command to run all those

Re: [Piglit] [PATCH 0/3] tests: intel blackhole render fixes

for the series: Reviewed-by: Dylan Baker <dy...@pnwbakers.com> Quoting Lionel Landwerlin (2018-04-06 03:25:13) > Hi, > > I've made a few mistake when sending the initial patch. I'm not quite > sure how I missed them :( > > Thanks, > > Lionel

Re: [Piglit] [PATCH] vkrunner: Add the script used to generate vr-format-table.h

Thanks for making the changes I requested! Reviewed-by: Dylan Baker <dy...@pnwbakers.com> Quoting Neil Roberts (2018-04-06 05:14:20) > Note that the script is not run automatically as part of the build > process and instead vr-format-table.h is checked into git. > --- > >

Re: [Piglit] [PATCH v2] framework: Add a vulkan tests profile

Looks good, thank you! Quoting Neil Roberts (2018-04-06 05:19:57) > This searches for files named *.vk_shader_test in the tests/vulkan > directory and runs them with VkRunner. > > Reviewed-by: Dylan Baker <dy...@pnwbakers.com> > > v2: Move VkShaderTest to p

[Piglit] [PATCH v2 34/35] Generate xml for builtin profiles

This creates and installs xml for all builtin profiles. Using these profiles I see startup times of ~1 second as opposed to more than 10 with current master, and runtimes that drop by ~1 minute. v2: - Fix compilation with make. v1 only worked with ninja. --- CMakeLists.txt | 8 +++-

Re: [Piglit] [PATCH 7/8] framework: Add a vulkan tests profile

-git a/tests/vulkan.py b/tests/vulkan.py > new file mode 100644 > index 0..e9c48f6a4 > --- /dev/null > +++ b/tests/vulkan.py > @@ -0,0 +1,30 @@ > +#!/usr/bin/env python > +# -*- coding: utf-8 -*- > +"""All Vulkan tests that come with piglit, using

Re: [Piglit] [PATCH 6/8] Import VkRunner

Quoting Neil Roberts (2018-04-04 15:56:53) [snip] > diff --git a/tests/vulkan/vkrunner/make-formats.py > b/tests/vulkan/vkrunner/make-formats.py > new file mode 100755 > index 0..6bb1030c0 > --- /dev/null > +++ b/tests/vulkan/vkrunner/make-formats.py > @@ -0,0 +1,93 @@ >

Re: [Piglit] [PATCH 00/35] Serialize profiles into XML at build time

Quoting Marek Olšák (2018-04-04 16:37:47) > Is this use case affected? > > piglit run --deqp-mustpass-list --process-isolation 0 -p gbm -c quick cts_gl45 > deqp_gles2 deqp_gles3 deqp_gles31 You could either run that as: piglit run --deqp-mustpass-list --process-isolation 0 -p gbm -c quick_gl

[Piglit] [PATCH v2 35/35] profile: use gz to compress profiles

This results in substantially smaller profiles and doesn't seem to affect runtime. v2: - install xml and xml.gz files. This is needed so that meta profiles will be installed. --- CMakeLists.txt | 4 ++-- framework/profile.py| 10 +- tests/CMakeLists.no_api.txt

Re: [Piglit] [PATCH 00/35] Serialize profiles into XML at build time

Quoting Dylan Baker (2018-04-04 15:26:48) > I don't expect everyone I've CC'd to give thorough review (or any > review), I've mostly CC'd people who I think would be interested in this > work, or who's work flow I might be altered by it. > > Piglit has struggled to cope with the

[Piglit] [PATCH 22/35] framework/profile: Add support for meta profiles

MetaProfiles are a single profile composed of more than one profile. This is designed to allow the old all-inclusive names to continue to exist without having to keep duplicates of each profile around. A metaprofile simply declares that it needs profiles a, b, and c and then runs those. This

[Piglit] [PATCH 01/35] update git ignore for this series

--- .gitignore | 4 1 file changed, 4 insertions(+) diff --git a/.gitignore b/.gitignore index c005a5a..0f4c952 100644 --- a/.gitignore +++ b/.gitignore @@ -30,3 +30,7 @@ __pycache__/ *.css *.html + +tests/*.xml +tests/*.xml.gz +!tests/*.meta.xml -- git-series 0.9.1

[Piglit] [PATCH 24/35] tests/quick: fix filtering of vs_in shader tests

This filter is incorrect as is, in that it can mask tests that it can remove tests it shouldn't, for example spec@arb_enhanced_layouts@arb_enhanced_layouts-transform-feedback-layout-qualifiers_vs_interface. Correcting this is problematic because it changes the way the filter works and filter will

[Piglit] [PATCH 04/35] framework/test: Use getter for altering PiglitBaseTest Command

This removes mutation of the class, and pushes the change out to run time instead of build time. This is better coding style, and is required for XML serialization, since paths need to be relative in the XML, and made absolute at runtime. --- framework/test/piglit_test.py | 5 - 1 file

[Piglit] [PATCH 34/35] Generate xml for builtin profiles

This creates and installs xml for all builtin profiles. Using these profiles I see startup times of ~1 second as opposed to more than 10 with current master, and runtimes that drop by ~1 minute. --- CMakeLists.txt | 8 ++- tests/CMakeLists.no_api.txt | 89

[Piglit] [PATCH 08/35] framework/test: add a test class for built-in constants

This has the same problem as asmparser tests. --- framework/test/piglit_test.py | 5 + 1 file changed, 5 insertions(+) diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py index c53c3e5..0986ab0 100644 --- a/framework/test/piglit_test.py +++

[Piglit] [PATCH 06/35] framework/profile: Add a __len__ method to TestProfile

This exposes a standard interface for getting the number of tests in a profile, which is itself nice. It will also allow us to encapsulate the differences between the various profiles added in this series. --- framework/profile.py | 14 +++--- 1 file changed, 7 insertions(+), 7

[Piglit] [PATCH 27/35] tests/glsl_parser_test.py: fix is_skip for serialized profiles

Currently is skip relies on runtime detection of which glslparsertest binaries are built, but we can't assume that at build time. Instead always assign the appropriate binary, and then check for the existence of that binary at run time. --- framework/test/glsl_parser_test.py| 20

[Piglit] [PATCH 21/35] profile: allow forcing python or xml loading

This will be used during build time to allow profiles that modify other profiles to load XML instead of rebuilding. --- framework/profile.py | 21 +++-- 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/framework/profile.py b/framework/profile.py index

[Piglit] [PATCH 00/35] Serialize profiles into XML at build time

runtime, and we've had to changes those assumptions. Dylan Baker (35): update git ignore for this series test/piglit_test: add ROOT_DIR variable framework/profile: Allow a group manager class to be overwritten framework/test: Use getter for altering PiglitBaseTest Command framework/test

[Piglit] [PATCH 19/35] framework/test: make CLProgramTester take relative paths

--- framework/test/piglit_test.py | 2 +- tests/cl.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py index 4fd825a..6511e0f 100644 --- a/framework/test/piglit_test.py +++

[Piglit] [PATCH 18/35] framework/test: make BuiltInConstantTest files relative

--- framework/test/piglit_test.py | 5 + tests/all.py | 26 +- 2 files changed, 18 insertions(+), 13 deletions(-) diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py index f5fd0c8..4fd825a 100644 ---

[Piglit] [PATCH 05/35] framework/test: expose required and excluded platforms

These need to be exposed for serialization. --- framework/test/piglit_test.py | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py index 3d5b156..ef3bfdc 100644 --- a/framework/test/piglit_test.py +++

[Piglit] [PATCH 09/35] tests: use BuiltInConstantsClass

--- tests/all.py | 55 ++--- 1 file changed, 27 insertions(+), 28 deletions(-) diff --git a/tests/all.py b/tests/all.py index fe4a14c..a50a4c3 100644 --- a/tests/all.py +++ b/tests/all.py @@ -21,7 +21,7 @@ from framework.driver_classifier import

[Piglit] [PATCH 30/35] tests/cl.py: fix out of tree serialization

--- tests/cl.py | 67 -- 1 file changed, 45 insertions(+), 22 deletions(-) diff --git a/tests/cl.py b/tests/cl.py index 56b3fc1..7d4e17d 100644 --- a/tests/cl.py +++ b/tests/cl.py @@ -104,47 +104,70 @@ with

[Piglit] [PATCH 20/35] profile: Add support for loading xml based profiles

Which includes de-serialization on demand. This will vastly reduce the amount of resident memory that is necessary to hold the tests since test instances are made on demand, and can be garbage collected as soon as the run in complete. --- framework/profile.py| 108

[Piglit] [PATCH 31/35] opengl.py: Remove exported gl extensions

This conflicts with the idea of serializing the tests at build time, since we can't assume that the build machine has the same features set as the machine that will run tests, we can't run such a filter. Since startup time is much faster and fast skip calculations are done at build time I hope

[Piglit] [PATCH 02/35] test/piglit_test: add ROOT_DIR variable

This variable always points to the root of the piglit directory, which contains the tests and generated_tests directories. --- framework/test/piglit_test.py | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py

[Piglit] [PATCH 23/35] py_modules: Add support for out of tree builds

For out of tree builds we need to point at a TESTS_DIR that is in tree, and a GENERATED_TESTS dir that is out of tree. At no other time do we need to do this. --- tests/py_modules/constants.py | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git

[Piglit] [PATCH 10/35] framework: use a class method for building test via parsing

Since the serialized tests wont need this path, build a generic constructor that this special constructor can leverage. --- framework/test/glsl_parser_test.py| 12 +++- tests/all.py | 2 +- unittests/framework/test/test_glsl_parser_test.py |

[Piglit] [PATCH 17/35] tests/all: Make asmparser tests path relative

--- tests/all.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/all.py b/tests/all.py index e0c7311..6aa45f0 100644 --- a/tests/all.py +++ b/tests/all.py @@ -353,13 +353,14 @@ for basedir in [TESTS_DIR, GENERATED_TESTS_DIR]: 'asmparsertest',

[Piglit] [PATCH 26/35] shader_tests: correctly generate xml during out of tree builds

--- framework/test/shader_test.py | 14 ++ tests/shader.py | 31 +++ 2 files changed, 37 insertions(+), 8 deletions(-) diff --git a/framework/test/shader_test.py b/framework/test/shader_test.py index dbad16b..719b92f 100644 ---

[Piglit] [PATCH 13/35] framework/test/piglit_test: make cl_concurrency always a boolean

Just a nice cleanup. --- framework/test/piglit_test.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py index 0986ab0..b5c9a53 100644 --- a/framework/test/piglit_test.py +++ b/framework/test/piglit_test.py @@

[Piglit] [PATCH 29/35] fix asmparser test serialization for out of tree builds

--- tests/glslparser.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/glslparser.py b/tests/glslparser.py index b7de9f6..fe2b3c5 100644 --- a/tests/glslparser.py +++ b/tests/glslparser.py @@ -59,7 +59,7 @@ for basedir in [TESTS_DIR, GENERATED_TESTS_DIR]:

[Piglit] [PATCH 32/35] Add script to serialize profiles to XML

This script loads a python module and creates an XML profiles. It will be used at build time to serialize the built-in profiles. It currently cannot serialize integrated suites (such as dEQP or IGC), but it could be extended to do so. --- tests/serializer.py | 149

[Piglit] [PATCH 16/35] framework/test: use relative paths for GLSLParser tests

--- framework/test/glsl_parser_test.py | 13 ++--- tests/all.py | 4 ++-- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/framework/test/glsl_parser_test.py b/framework/test/glsl_parser_test.py index 46344a8..35eadfe 100644 ---

[Piglit] [PATCH 14/35] framework/test: Add class for cl-program-tester

These tests need some special handling for serializing, since they take file paths as arguments. --- framework/test/piglit_test.py | 35 +-- tests/cl.py | 6 +++--- 2 files changed, 28 insertions(+), 13 deletions(-) diff --git

[Piglit] [PATCH 03/35] framework/profile: Allow a group manager class to be overwritten

This is useful for classes like the coming BuiltInConstantsTest class, where just one or two tests need to change the test class. --- framework/profile.py | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/framework/profile.py b/framework/profile.py index d77ac09..b6deed0

[Piglit] [PATCH 11/35] framework: do the same for shader test

--- framework/test/shader_test.py| 46 +++-- tests/all.py | 4 +- unittests/framework/test/test_shader_test.py | 20 - 3 files changed, 38 insertions(+), 32 deletions(-) diff --git a/framework/test/shader_test.py

[Piglit] [PATCH 07/35] framework: Use custom class for ASM parser tests

This is going to make serialization easier, since asmparser tests take a file argument, which must be relative at build time, but absolute at run time. --- framework/test/piglit_test.py | 14 ++ tests/all.py | 6 -- 2 files changed, 18 insertions(+), 2

[Piglit] [PATCH 35/35] profile: use gz to compress profiles

This results in substantially smaller profiles and doesn't seem to affect runtime. --- CMakeLists.txt | 4 ++-- framework/profile.py| 8 tests/CMakeLists.no_api.txt | 6 +++--- tests/serializer.py | 4 +++- 4 files changed, 12 insertions(+), 10 deletions(-)

[Piglit] [PATCH 33/35] tests: Add script to find all hand written test files

Including asmparser, glslparser, shader, and program tests, which will be used by cmake to regenerate the appropriate xml whenever a new test is added. --- tests/find_static_tests.py | 74 +++- 1 file changed, 74 insertions(+) create mode 100644

[Piglit] [PATCH 12/35] framework/test: Split multishader too

--- framework/test/shader_test.py| 63 ++--- tests/all.py | 2 +- unittests/framework/test/test_shader_test.py | 17 +- 3 files changed, 59 insertions(+), 23 deletions(-) diff --git a/framework/test/shader_test.py

[Piglit] [PATCH 28/35] fix glslparser test for out of tree builds

--- framework/test/glsl_parser_test.py | 19 +-- tests/glslparser.py| 13 +++-- 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/framework/test/glsl_parser_test.py b/framework/test/glsl_parser_test.py index 88646c9..8840c2d 100644 ---

Re: [Piglit] [PATCH] tests: add nvidia conservative rasterization tests

Quoting Pending Chaos (2018-03-31 03:13:58) > GL_NV_conservative_raster, GL_NV_conservative_raster_dilate and > GL_NV_conservative_raster_pre_snap_triangles are three separate extensions. Thanks for clarifying :) Dylan signature.asc Description: signature

Re: [Piglit] [PATCH] tests: add nvidia conservative rasterization tests

Quoting Rhys Perry (2018-03-22 06:43:25) > Adds tests for GL_NV_conservative_raster, GL_NV_conservative_raster_dilate > and GL_NV_conservative_raster_pre_snap_triangles. > --- > tests/all.py | 20 ++ > tests/spec/CMakeLists.txt | 3

Re: [Piglit] [PATCH] Add support for Crucible

late_profile(): > +profile = TestProfile() > +case_file = gen_caselist_txt(crucible_bin) > +with open(case_file, 'r') as caselist_file: > +for i, line in enumerate(caselist_file): > +case = line.rstrip() > +piglit_name = case.replace('.

Re: [Piglit] [PATCH v3 1/1] framework: change one missed core.checkDir call to core.check_dir

Quoting Petri Latvala (2018-03-26 02:51:08) > > > On 03/23/2018 06:11 PM, Dylan Baker wrote: > > Quoting Petri Latvala (2018-03-23 03:11:10) > >> Commit 5e699e402a8b ("framework: rename core.checkDir to > >> core.check_dir") missed one call to che

Re: [Piglit] [PATCH v3 1/1] framework: change one missed core.checkDir call to core.check_dir

ixed indentation > > Signed-off-by: Petri Latvala <petri.latv...@intel.com> > Cc: Dylan Baker <dy...@pnwbakers.com> > Cc: Jari Tahvanainen <jari.tahvanai...@intel.com> > --- > > The indent was with spaces, but just incorrectly indented. > > framewo

Re: [Piglit] [PATCH v2 1/1] framework: change one missed core.checkDir call to core.check_dir

t; Signed-off-by: Petri Latvala <petri.latv...@intel.com> > Cc: Dylan Baker <dy...@pnwbakers.com> > Cc: Jari Tahvanainen <jari.tahvanai...@intel.com> > --- > framework/programs/summary.py | 8 +++- > 1 file changed, 7 insertions(+), 1 deletion(-) > > diff

Re: [Piglit] [PATCH] cmake: Use WAYLAND_LDFLAGS instead of WAYLAND_LIBRARIES.

t(APPEND UTIL_GL_LIBS ${WAYLAND_LIBRARIES}) > + list(APPEND UTIL_GL_LIBS ${WAYLAND_LDFLAGS}) > endif() > > endif(${CMAKE_SYSTEM_NAME} MATCHES "Linux") > -- > 2.14.3 > > ___ > Piglit mailin

Re: [Piglit] [PATCH] tests: Added a new GLES test profile for the Khronos CTS runner

Reviewed-by: Dylan Baker <dy...@pnwbakers.com> Quoting Andres Gomez (2018-02-13 05:24:10) > OpenGL GLES*-CTS case lists were renamed to KHR-GLES* in the upstream > repository. > > We want to keep the existing profiles so we are able to keep running > the caselists from

Re: [Piglit] [PATCH] degenerate-prims: fix incorrect return value from test_prim()

Oops, Reviewed-by: Dylan Baker <dy...@pnwbakers.com> Quoting Brian Paul (2018-02-13 13:26:18) > The function is supposed to return a piglit_result, not true/false. > Fixes incorrect 'fail' results. > --- > tests/general/degenerate-prims.c | 2 +- > 1 file changed, 1 ins

Re: [Piglit] [PATCH] AMD_performance_monitor: add VC4 specific tests

If there's no other changes required I don't need to see a v2 just to fix all.py, I'm not really qualified to review the patch otherwise though. Dylan Quoting Boris Brezillon (2018-02-08 05:39:04) > Hi Dylan, > > On Wed, 07 Feb 2018 10:38:28 -0800 > Dylan Baker <dy...@pnwba

Re: [Piglit] [PATCH] AMD_performance_monitor: add VC4 specific tests

You'll need to add this to all.py. Quoting Boris Brezillon (2018-02-07 00:19:34) > From: Boris Brezillon > > This adds a specific test for the VC4 GPU. > Right now, it only checks that FEP-valid-quads and > QPU-total-clk-cycles-waiting-TMU are consistent

Re: [Piglit] [PATCH v4] travis: add docker based cmake build job

I haven't looked too closely at this, but I like the idea of using docker to allow running the cmake based job. Acked-by: Dylan Baker <dy...@pnwbakers.com> Quoting Andres Gomez (2018-01-31 13:11:52) > Until now we were only running the python unit tests. > > It seems desirabl

Re: [Piglit] [PATCH] all.py: fix extension/version testing when wflinfo fails

Quoting Brian Paul (2018-02-02 11:46:22) > On 02/02/2018 11:57 AM, Dylan Baker wrote: > > Quoting Brian Paul (2018-02-02 09:35:21) > >> If wflinfo is not installed or fails, the wfl_info.gl_extensions value > >> will be the empty set. Similarly, the gl

Re: [Piglit] Log crashed subtest V2

Quoting Dylan Baker (2018-01-30 09:55:06) > for the series: > Reviewed-by: Dylan Baker <dy...@pnwbakers.com> > > Quoting Fabian Bieler (2018-01-28 16:22:10) > > The idea is to convert tests with subtests to either use > > piglit_run_selected_subtests or call piglit_

Re: [Piglit] [PATCH] all.py: fix extension/version testing when wflinfo fails

_name[:5] == "glsl-": > # The directory is a GLSL version > version = float(dir_name[5:]) > -return version <= wfl_info.glsl_version > +return wfl_info.glsl_version == None or version <= > wfl_info.glsl_version > else: > # The directory is something else. Don't skip it. > return True With the nits above fixed, Reviewed-by: Dylan Baker <dy...@pnwbakers.com> signature.asc Description: signature ___ Piglit mailing list Piglit@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] Log crashed subtest V2

for the series: Reviewed-by: Dylan Baker <dy...@pnwbakers.com> Quoting Fabian Bieler (2018-01-28 16:22:10) > The idea is to convert tests with subtests to either use > piglit_run_selected_subtests or call piglit_register_subtests at the start of > the test. Either approach enu

Re: [Piglit] [PATCH 01/15] tests/util: Add config value to enumerate subtests

limited resources, ...) aren't" > and not "test shouldn't be run to begin with (e.g. because it's not included > in the current test suite)" for which I thought "not run" is the appropriate > status. > > Fabian > > On Fri, Jan 26, 2018, at 6:53 PM, D

Re: [Piglit] [PATCH 01/15] tests/util: Add config value to enumerate subtests

too. > > Note that I leveraged piglit_run_selected_subtests for gl-2.1-pbo, > fbo-incomplete, degenerate-prims and linestipple and used > piglit_register_subtests in fbo-storage-formats. > > Regards (and sorry again) > Fabian > > On 2018-01-23 02:22, Dylan Baker wrote: &g

Re: [Piglit] [PATCH 01/15] tests/util: Add config value to enumerate subtests

No worries. I'd rather get this right the first time before we convert 100+ tests than afterwards :) I also appreciate that you've converted some of the tests to use piglit_run_selected_subtests. I've lamented with others that it's unfortunate that we've never really used that feature, since it

[Piglit] [PATCH 08/15] tests/fbo-storage-formats: Always print the same number of subtests

!skip -> skip won't show up in the regressions/fixes/etc lists anyway, and this means that the output will always be the same. --- tests/fbo/fbo-storage-formats.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/tests/fbo/fbo-storage-formats.c b/tests/fbo/fbo-storage-formats.c index

[Piglit] [PATCH 12/15] tests: always report subtest results for degenerate-prims

--- tests/general/degenerate-prims.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/tests/general/degenerate-prims.c b/tests/general/degenerate-prims.c index 7ed6093..d2bb7d6 100644 --- a/tests/general/degenerate-prims.c +++ b/tests/general/degenerate-prims.c @@ -63,10

[Piglit] [PATCH 15/15] Add series todo

--- todo | 108 - 1 file changed, 108 insertions(+) create mode 100644 todo diff --git a/todo b/todo new file mode 100644 index 000..c869582 --- /dev/null +++ b/todo @@ -0,0 +1,108 @@ +tests/cl/api/set-kernel-arg.c

[Piglit] [PATCH 01/15] tests/util: Add config value to enumerate subtests

This adds a new member to the GL config struct for informing the python framework the number and order of subtests that will be run (if any). If this value is unset then no subtests are expected. --- tests/util/piglit-framework-gl.c | 13 + tests/util/piglit-framework-gl.h | 11

[Piglit] [PATCH 06/15] tests: enumerate subtests in gl-2.1-gbo test

This gives us something to test the python part against. Signed-off-by: Dylan Baker <dylan.c.ba...@intel.com> --- tests/spec/gl-2.1/pbo.c | 12 1 file changed, 12 insertions(+) diff --git a/tests/spec/gl-2.1/pbo.c b/tests/spec/gl-2.1/pbo.c index 83dc1c4..81d4f64 100644 --- a

[Piglit] [PATCH 13/15] tests: enumerate subtests in degenerate-prims

--- tests/general/degenerate-prims.c | 59 ++--- 1 file changed, 41 insertions(+), 18 deletions(-) diff --git a/tests/general/degenerate-prims.c b/tests/general/degenerate-prims.c index d2bb7d6..6a73b65 100644 --- a/tests/general/degenerate-prims.c +++

[Piglit] [PATCH 07/15] tests: enumerate subtests in fbo-incomplete

Signed-off-by: Dylan Baker <dylan.c.ba...@intel.com> --- tests/fbo/fbo-incomplete.cpp | 12 1 file changed, 12 insertions(+) diff --git a/tests/fbo/fbo-incomplete.cpp b/tests/fbo/fbo-incomplete.cpp index 8cde6d2..8d16ec1 100644 --- a/tests/fbo/fbo-incomplete.cpp +++ b/tests/f

[Piglit] [PATCH 05/15] framework: if a test with subtests crashes mark the offending subtest

This relies on the fact that subtests are guaranteed to be ordered to mark the crashing subtest as such. This ensures that the correct status will be propagated up the totals tree. --- framework/test/base.py| 8 ++- unittests/framework/test/test_base.py | 36

[Piglit] [PATCH 03/15] framework: use statuses instead of relying on string conversions

This is a small cleanup, but we need to touch this code further for subtest enumeration work, and we'll want to use statuses there too. --- framework/test/base.py | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/framework/test/base.py b/framework/test/base.py index

[Piglit] [PATCH 09/15] tests/fbo-storage-formats: Make subtest names predictable

of the test apart from the subtest name, and name the subtest name just the format name. This also makes the name the same when the test fails and when in passes. Signed-off-by: Dylan Baker <dylan.c.ba...@intel.com> --- tests/fbo/fbo-storage-formats.c | 9 - 1 file changed, 4 inse

[Piglit] [PATCH 11/15] tests/fbo-storage-formats: enumerate subtests

--- tests/fbo/fbo-storage-formats.c | 41 ++ 1 file changed, 27 insertions(+), 14 deletions(-) diff --git a/tests/fbo/fbo-storage-formats.c b/tests/fbo/fbo-storage-formats.c index 7f13684..ee22883 100644 --- a/tests/fbo/fbo-storage-formats.c +++

[Piglit] [PATCH 10/15] tests/fbo-storage-formats: print subtest result for skip too

Because we always want to have the same number of subtests printed, or they'll show up as "NOTRUN" instead of "SKIP" in the summary. Signed-off-by: Dylan Baker <dylan.c.ba...@intel.com> --- tests/fbo/fbo-storage-formats.c | 4 +++- 1 file changed, 3 insertions(+),

[Piglit] [PATCH 14/15] tests: enumerate subtests in linestipple

--- tests/general/linestipple.c | 24 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/tests/general/linestipple.c b/tests/general/linestipple.c index ba0924e..6626b3c 100644 --- a/tests/general/linestipple.c +++ b/tests/general/linestipple.c @@ -28,14 +28,6

[Piglit] [PATCH 02/15] framework: use an OrderedDict for subtest storage

This makes the order of the subtests deterministic. Since piglit test binaries are expected to enumerate subtests in the same order that they'll run them this will allow us to know which subtest crashed when there is a crash. --- framework/results.py | 2 +- 1 file changed, 1 insertion(+), 1

[Piglit] [PATCH 04/15] framework: add support for parsing subtest enumeration

This adds support for enumerating subtests to the python layer. When it sees this it sets each subtest to notrun. This allows the python framework to report that tests didn't run when they were expected to. Signed-off-by: Dylan Baker <dylan.c.ba...@intel.com> --- framework/test/piglit_t

Re: [Piglit] [PATCH] framework: Propagate crash to subtest.

Quoting Fabian Bieler (2018-01-20 05:38:45) > If a subtest crashed it and all following subtests have the result notrun. > Only the test as a whole has the result crash. > This commit propagates the crash to the correct subtest. > > Note that if the crash happened after the last subtest it is

Re: [Piglit] [RFC 1/9] util: Add a function for enumerating subtests

Dylan Baker (2018-01-19 16:25:56) > This function takes one or more subtests as strings and returns a JSON > structure that the python framework can consume. > > Signed-off-by: Dylan Baker <dylan.c.ba...@intel.com> > --- > tests/util/piglit-util.c | 29 +++

[Piglit] [PATCH 5/9] tests/fbo-storage-formats: Always print the same number of subtests

!skip -> skip won't show up in the regressions/fixes/etc lists anyway, and this means that the output will always be the same. --- tests/fbo/fbo-storage-formats.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/tests/fbo/fbo-storage-formats.c b/tests/fbo/fbo-storage-formats.c index

[Piglit] [PATCH 3/9] framework: add support for parsing subtest enumeration

This adds support for enumerating subtests to the python layer. When it sees this it sets each subtest to notrun. This allows the python framework to report that tests didn't run when they were expected to. Signed-off-by: Dylan Baker <dylan.c.ba...@intel.com> --- framework/test/piglit_t

[Piglit] [PATCH 8/9] tests/fbo-storage-formats: enumerate subtests

--- tests/fbo/fbo-storage-formats.c | 22 ++ 1 file changed, 22 insertions(+) diff --git a/tests/fbo/fbo-storage-formats.c b/tests/fbo/fbo-storage-formats.c index 4db990d..a11335b 100644 --- a/tests/fbo/fbo-storage-formats.c +++ b/tests/fbo/fbo-storage-formats.c @@ -243,9

<    1   2   3   4   5   6   7   8   9   10   >