Re: [Piglit] [PATCH 00/35] Serialize profiles into XML at build time

Is this use case affected? piglit run --deqp-mustpass-list --process-isolation 0 -p gbm -c quick cts_gl45 deqp_gles2 deqp_gles3 deqp_gles31 Yes, that is just 1 command to run all those test suites at the same time. I use my personal "deqp" piglit branch that also disables process isolation for

[Piglit] [PATCH 7/8] framework: Add a vulkan tests profile

This searches for files named *.vk_shader_test in the tests/vulkan directory and runs them with VkRunner. --- framework/test/vk_shader_test.py | 39 +++ tests/vulkan.py | 30 ++ 2 files changed, 69 insertions(+)

[Piglit] [PATCH 8/8] vulkan: Add some tests for glsl450 builtin functions using doubles

--- .../glsl450/face-forward-double.vk_shader_test | 88 ++ tests/vulkan/glsl450/frexp-double.vk_shader_test | 61 +++ tests/vulkan/glsl450/isinf-double.vk_shader_test | 81 tests/vulkan/glsl450/reflect-double.vk_shader_test | 55

[Piglit] [PATCH 5/8] framework: Add a Vulkan platform

This defaults to building only on Linux as with the GLES platforms because later patches add code using fork/exec which will need porting to compile on Windows. --- CMakeLists.txt | 7 +++ cmake/piglit_util.cmake| 2 +- cmake/target_api/CMakeLists.txt

[Piglit] [PATCH 4/8] util: Move the half float functions from util-gl to util

They will be equally useful on Vulkan. --- tests/util/piglit-util-gl.c | 90 - tests/util/piglit-util-gl.h | 29 --- tests/util/piglit-util.c| 89 tests/util/piglit-util.h| 28

[Piglit] [PATCH 0/8] Vulkan testing with VkRunner

This series sets up a framework for testing Vulkan in Piglit. The main thing is to import a tool like shader_runner called VkRunner. There are also 5 actual tests which mainly serve as examples. The code for VkRunner was developed as a standalone project here: https://github.com/igalia/vkrunner

[Piglit] [PATCH 2/8] util: Add a piglit_fatal utility

This prints a formatted error and then aborts. --- tests/util/piglit-util.c | 16 tests/util/piglit-util.h | 2 ++ 2 files changed, 18 insertions(+) diff --git a/tests/util/piglit-util.c b/tests/util/piglit-util.c index ed7d21c95..a79db3fb7 100644 --- a/tests/util/piglit-util.c

[Piglit] [PATCH 1/8] util: Add a PIGLIT_STRINGIFY macro

--- tests/util/piglit-util.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/util/piglit-util.h b/tests/util/piglit-util.h index 0a5eab810..3dfb5d6c5 100644 --- a/tests/util/piglit-util.h +++ b/tests/util/piglit-util.h @@ -162,6 +162,8 @@ piglit_register_subtests(const char *names[]);

[Piglit] [PATCH 3/8] util: Add a libutil_vulkan library

This is currently no different from libutil but it is added anyway to match the other APIs. --- tests/util/CMakeLists.vulkan.txt | 15 +++ tests/util/piglit-util-vulkan.c | 1 + 2 files changed, 16 insertions(+) create mode 100644 tests/util/CMakeLists.vulkan.txt create mode

[Piglit] [PATCH v2 35/35] profile: use gz to compress profiles

This results in substantially smaller profiles and doesn't seem to affect runtime. v2: - install xml and xml.gz files. This is needed so that meta profiles will be installed. --- CMakeLists.txt | 4 ++-- framework/profile.py| 10 +- tests/CMakeLists.no_api.txt

Re: [Piglit] [PATCH 00/35] Serialize profiles into XML at build time

Quoting Dylan Baker (2018-04-04 15:26:48) > I don't expect everyone I've CC'd to give thorough review (or any > review), I've mostly CC'd people who I think would be interested in this > work, or who's work flow I might be altered by it. > > Piglit has struggled to cope with the growing number of

[Piglit] [PATCH 22/35] framework/profile: Add support for meta profiles

MetaProfiles are a single profile composed of more than one profile. This is designed to allow the old all-inclusive names to continue to exist without having to keep duplicates of each profile around. A metaprofile simply declares that it needs profiles a, b, and c and then runs those. This

[Piglit] [PATCH 01/35] update git ignore for this series

--- .gitignore | 4 1 file changed, 4 insertions(+) diff --git a/.gitignore b/.gitignore index c005a5a..0f4c952 100644 --- a/.gitignore +++ b/.gitignore @@ -30,3 +30,7 @@ __pycache__/ *.css *.html + +tests/*.xml +tests/*.xml.gz +!tests/*.meta.xml -- git-series 0.9.1

[Piglit] [PATCH 24/35] tests/quick: fix filtering of vs_in shader tests

This filter is incorrect as is, in that it can mask tests that it can remove tests it shouldn't, for example spec@arb_enhanced_layouts@arb_enhanced_layouts-transform-feedback-layout-qualifiers_vs_interface. Correcting this is problematic because it changes the way the filter works and filter will

[Piglit] [PATCH 04/35] framework/test: Use getter for altering PiglitBaseTest Command

This removes mutation of the class, and pushes the change out to run time instead of build time. This is better coding style, and is required for XML serialization, since paths need to be relative in the XML, and made absolute at runtime. --- framework/test/piglit_test.py | 5 - 1 file

[Piglit] [PATCH 34/35] Generate xml for builtin profiles

This creates and installs xml for all builtin profiles. Using these profiles I see startup times of ~1 second as opposed to more than 10 with current master, and runtimes that drop by ~1 minute. --- CMakeLists.txt | 8 ++- tests/CMakeLists.no_api.txt | 89

[Piglit] [PATCH 08/35] framework/test: add a test class for built-in constants

This has the same problem as asmparser tests. --- framework/test/piglit_test.py | 5 + 1 file changed, 5 insertions(+) diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py index c53c3e5..0986ab0 100644 --- a/framework/test/piglit_test.py +++

[Piglit] [PATCH 06/35] framework/profile: Add a __len__ method to TestProfile

This exposes a standard interface for getting the number of tests in a profile, which is itself nice. It will also allow us to encapsulate the differences between the various profiles added in this series. --- framework/profile.py | 14 +++--- 1 file changed, 7 insertions(+), 7

[Piglit] [PATCH 27/35] tests/glsl_parser_test.py: fix is_skip for serialized profiles

Currently is skip relies on runtime detection of which glslparsertest binaries are built, but we can't assume that at build time. Instead always assign the appropriate binary, and then check for the existence of that binary at run time. --- framework/test/glsl_parser_test.py| 20

[Piglit] [PATCH 21/35] profile: allow forcing python or xml loading

This will be used during build time to allow profiles that modify other profiles to load XML instead of rebuilding. --- framework/profile.py | 21 +++-- 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/framework/profile.py b/framework/profile.py index

[Piglit] [PATCH 00/35] Serialize profiles into XML at build time

I don't expect everyone I've CC'd to give thorough review (or any review), I've mostly CC'd people who I think would be interested in this work, or who's work flow I might be altered by it. Piglit has struggled to cope with the growing number of tests that it contains, especially with startup

[Piglit] [PATCH 19/35] framework/test: make CLProgramTester take relative paths

--- framework/test/piglit_test.py | 2 +- tests/cl.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py index 4fd825a..6511e0f 100644 --- a/framework/test/piglit_test.py +++

[Piglit] [PATCH 18/35] framework/test: make BuiltInConstantTest files relative

--- framework/test/piglit_test.py | 5 + tests/all.py | 26 +- 2 files changed, 18 insertions(+), 13 deletions(-) diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py index f5fd0c8..4fd825a 100644 ---

[Piglit] [PATCH 05/35] framework/test: expose required and excluded platforms

These need to be exposed for serialization. --- framework/test/piglit_test.py | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py index 3d5b156..ef3bfdc 100644 --- a/framework/test/piglit_test.py +++

[Piglit] [PATCH 09/35] tests: use BuiltInConstantsClass

--- tests/all.py | 55 ++--- 1 file changed, 27 insertions(+), 28 deletions(-) diff --git a/tests/all.py b/tests/all.py index fe4a14c..a50a4c3 100644 --- a/tests/all.py +++ b/tests/all.py @@ -21,7 +21,7 @@ from framework.driver_classifier import

[Piglit] [PATCH 30/35] tests/cl.py: fix out of tree serialization

--- tests/cl.py | 67 -- 1 file changed, 45 insertions(+), 22 deletions(-) diff --git a/tests/cl.py b/tests/cl.py index 56b3fc1..7d4e17d 100644 --- a/tests/cl.py +++ b/tests/cl.py @@ -104,47 +104,70 @@ with

[Piglit] [PATCH 20/35] profile: Add support for loading xml based profiles

Which includes de-serialization on demand. This will vastly reduce the amount of resident memory that is necessary to hold the tests since test instances are made on demand, and can be garbage collected as soon as the run in complete. --- framework/profile.py| 108

[Piglit] [PATCH 31/35] opengl.py: Remove exported gl extensions

This conflicts with the idea of serializing the tests at build time, since we can't assume that the build machine has the same features set as the machine that will run tests, we can't run such a filter. Since startup time is much faster and fast skip calculations are done at build time I hope

[Piglit] [PATCH 02/35] test/piglit_test: add ROOT_DIR variable

This variable always points to the root of the piglit directory, which contains the tests and generated_tests directories. --- framework/test/piglit_test.py | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py

[Piglit] [PATCH 23/35] py_modules: Add support for out of tree builds

For out of tree builds we need to point at a TESTS_DIR that is in tree, and a GENERATED_TESTS dir that is out of tree. At no other time do we need to do this. --- tests/py_modules/constants.py | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git

[Piglit] [PATCH 10/35] framework: use a class method for building test via parsing

Since the serialized tests wont need this path, build a generic constructor that this special constructor can leverage. --- framework/test/glsl_parser_test.py| 12 +++- tests/all.py | 2 +- unittests/framework/test/test_glsl_parser_test.py |

[Piglit] [PATCH 17/35] tests/all: Make asmparser tests path relative

--- tests/all.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/all.py b/tests/all.py index e0c7311..6aa45f0 100644 --- a/tests/all.py +++ b/tests/all.py @@ -353,13 +353,14 @@ for basedir in [TESTS_DIR, GENERATED_TESTS_DIR]: 'asmparsertest',

[Piglit] [PATCH 26/35] shader_tests: correctly generate xml during out of tree builds

--- framework/test/shader_test.py | 14 ++ tests/shader.py | 31 +++ 2 files changed, 37 insertions(+), 8 deletions(-) diff --git a/framework/test/shader_test.py b/framework/test/shader_test.py index dbad16b..719b92f 100644 ---

[Piglit] [PATCH 13/35] framework/test/piglit_test: make cl_concurrency always a boolean

Just a nice cleanup. --- framework/test/piglit_test.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py index 0986ab0..b5c9a53 100644 --- a/framework/test/piglit_test.py +++ b/framework/test/piglit_test.py @@

[Piglit] [PATCH 29/35] fix asmparser test serialization for out of tree builds

--- tests/glslparser.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/glslparser.py b/tests/glslparser.py index b7de9f6..fe2b3c5 100644 --- a/tests/glslparser.py +++ b/tests/glslparser.py @@ -59,7 +59,7 @@ for basedir in [TESTS_DIR, GENERATED_TESTS_DIR]:

[Piglit] [PATCH 32/35] Add script to serialize profiles to XML

This script loads a python module and creates an XML profiles. It will be used at build time to serialize the built-in profiles. It currently cannot serialize integrated suites (such as dEQP or IGC), but it could be extended to do so. --- tests/serializer.py | 149

[Piglit] [PATCH 16/35] framework/test: use relative paths for GLSLParser tests

--- framework/test/glsl_parser_test.py | 13 ++--- tests/all.py | 4 ++-- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/framework/test/glsl_parser_test.py b/framework/test/glsl_parser_test.py index 46344a8..35eadfe 100644 ---

[Piglit] [PATCH 14/35] framework/test: Add class for cl-program-tester

These tests need some special handling for serializing, since they take file paths as arguments. --- framework/test/piglit_test.py | 35 +-- tests/cl.py | 6 +++--- 2 files changed, 28 insertions(+), 13 deletions(-) diff --git

[Piglit] [PATCH 03/35] framework/profile: Allow a group manager class to be overwritten

This is useful for classes like the coming BuiltInConstantsTest class, where just one or two tests need to change the test class. --- framework/profile.py | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/framework/profile.py b/framework/profile.py index d77ac09..b6deed0

[Piglit] [PATCH 11/35] framework: do the same for shader test

--- framework/test/shader_test.py| 46 +++-- tests/all.py | 4 +- unittests/framework/test/test_shader_test.py | 20 - 3 files changed, 38 insertions(+), 32 deletions(-) diff --git a/framework/test/shader_test.py

[Piglit] [PATCH 07/35] framework: Use custom class for ASM parser tests

This is going to make serialization easier, since asmparser tests take a file argument, which must be relative at build time, but absolute at run time. --- framework/test/piglit_test.py | 14 ++ tests/all.py | 6 -- 2 files changed, 18 insertions(+), 2

[Piglit] [PATCH 35/35] profile: use gz to compress profiles

This results in substantially smaller profiles and doesn't seem to affect runtime. --- CMakeLists.txt | 4 ++-- framework/profile.py| 8 tests/CMakeLists.no_api.txt | 6 +++--- tests/serializer.py | 4 +++- 4 files changed, 12 insertions(+), 10 deletions(-)

[Piglit] [PATCH 33/35] tests: Add script to find all hand written test files

Including asmparser, glslparser, shader, and program tests, which will be used by cmake to regenerate the appropriate xml whenever a new test is added. --- tests/find_static_tests.py | 74 +++- 1 file changed, 74 insertions(+) create mode 100644

[Piglit] [PATCH 12/35] framework/test: Split multishader too

--- framework/test/shader_test.py| 63 ++--- tests/all.py | 2 +- unittests/framework/test/test_shader_test.py | 17 +- 3 files changed, 59 insertions(+), 23 deletions(-) diff --git a/framework/test/shader_test.py

[Piglit] [PATCH 28/35] fix glslparser test for out of tree builds

--- framework/test/glsl_parser_test.py | 19 +-- tests/glslparser.py| 13 +++-- 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/framework/test/glsl_parser_test.py b/framework/test/glsl_parser_test.py index 88646c9..8840c2d 100644 ---

Re: [Piglit] [PATCH] cl: Add test for call stack realignment

On Tue, 2018-04-03 at 18:03 -0400, Matt Arsenault wrote: > ping > > > On Mar 29, 2018, at 10:35, Matt Arsenault wrote: > > > > --- > > tests/cl/program/execute/realign-stack.cl | 96 > > +++ > > 1 file changed, 96 insertions(+) > > create mode

Re: [Piglit] [PATCH 1/3] arb_get_texture_sub_image: fix expected error when querying a level which hasn't been explicitly defined

On Wed, 2018-04-04 at 15:04 +, Anthony Pesch wrote: > Hey Juan, > > The change from INVALID_OPERATION to INVALID_VALUE isn't because the level > value is invalid. > > The level is valid, however, the level hasn't been explicitly defined so it > should have a default width and height of 0,

Re: [Piglit] [PATCH 1/3] arb_get_texture_sub_image: fix expected error when querying a level which hasn't been explicitly defined

Hey Juan, The change from INVALID_OPERATION to INVALID_VALUE isn't because the level value is invalid. The level is valid, however, the level hasn't been explicitly defined so it should have a default width and height of 0, making the 8x8 query invalid. From the OpenGL 4.6 spec, 8.22 Texture

Re: [Piglit] [PATCH 2/3] arb_get_texture_sub_image: fix depth parameter when performing zero-sized texture queries

On Wed, 2018-03-28 at 11:15 -0400, Anthony Pesch wrote: > From: Anthony Pesch > > Change zero-sized texture queries to pass a valid depth parameter, otherwise > they should error as per the OpenGL 4.6 Core spec: > > "An INVALID_VALUE error is generated if the effective target

Re: [Piglit] [PATCH 1/3] arb_get_texture_sub_image: fix expected error when querying a level which hasn't been explicitly defined

On Wed, 2018-03-28 at 11:15 -0400, Anthony Pesch wrote: > From: Anthony Pesch > > Change expected error from INVALID_OPERATION to INVALID_VALUE when querying > a level which hasn't been explicitly defined. This is a valid operation, the > error set should be due to the