[Piglit] [PATCH 1/3] README: Update the list of test profiles

[Rhys Kidd]

Signed-off-by: Rhys Kidd 
---
 README | 30 ++
 1 file changed, 30 insertions(+)

diff --git a/README b/README
index 43bdfcae7..cb49516b7 100644
--- a/README
+++ b/README
@@ -386,6 +386,9 @@ glslparser.py
 shader.py
A subset of all.py which runs only shader tests
 
+no_error.py
+   A modified version of the test list run as khr_no_error variants
+
 
 4.2 OpenCL Tests
 
@@ -407,6 +410,9 @@ xts.py
 igt.py
Support for running Intel-gpu-tools test suite using piglit.
 
+deqp_egl.py
+   Support for running dEQP's EGL profile with piglit.
+
 deqp_gles2.py
Support for running deQP's gles2 profile with piglit.
 
@@ -416,6 +422,30 @@ deqp_gles3.py
 deqp_gles31.py
Support for running deQP's gles3.1 profile with piglit.
 
+deqp_vk.py
+   Support for running the official Khronos Vulkan CTS profile with piglit.
+
+khr_gl.py
+   Support for running the open source Khronos OpenGL CTS tests with 
piglit.
+
+khr_gl45.py
+   Support for running the open source Khronos OpenGL 4.5 CTS tests with 
piglit.
+
+cts_gl.py
+   Support for running the closed source Khronos OpenGL CTS tests with 
piglit.
+
+cts_gl45.py
+   Support for running the closed source Khronos OpenGL 4.5 CTS tests with 
piglit.
+
+cts_gl.py
+   Support for running the closed source Khronos GLES CTS tests with 
piglit.
+
+oglconform.py
+   Support for running sub-test of the Intel oglconform test suite with 
piglit.
+
+es3conform.py
+   Support for running a subset of Khronos ES 3.0 Conformance suite with 
piglit.
+
 
 5. How to write tests
 -
-- 
2.11.0

___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

[Piglit] [PATCH 2/3] README: Correct typo deQP -> dEQP

[Rhys Kidd]

Signed-off-by: Rhys Kidd 
---
 README | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/README b/README
index cb49516b7..56ea52b62 100644
--- a/README
+++ b/README
@@ -414,13 +414,13 @@ deqp_egl.py
Support for running dEQP's EGL profile with piglit.
 
 deqp_gles2.py
-   Support for running deQP's gles2 profile with piglit.
+   Support for running dEQP's gles2 profile with piglit.
 
 deqp_gles3.py
-   Support for running deQP's gles3 profile with piglit.
+   Support for running dEQP's gles3 profile with piglit.
 
 deqp_gles31.py
-   Support for running deQP's gles3.1 profile with piglit.
+   Support for running dEQP's gles3.1 profile with piglit.
 
 deqp_vk.py
Support for running the official Khronos Vulkan CTS profile with piglit.
-- 
2.11.0

___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

[Piglit] [PATCH 3/3] README: Make initial smoke tests steps internally consistent

[Rhys Kidd]

No other stage of the suggested initial sanity.py test or other
tests uses a folder structure with *.results.

Use a consistent naming approach throughout the initial README to
make initial setup clearer.

Signed-off-by: Rhys Kidd 
---
 README | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/README b/README
index 56ea52b62..c6961cfac 100644
--- a/README
+++ b/README
@@ -261,12 +261,12 @@ See also section 4.
 
 To create some nice formatted test summaries, run
 
-  $ ./piglit summary html summary/sanity results/sanity.results
+  $ ./piglit summary html summary/sanity results/sanity
 
 Hint: You can combine multiple test results into a single summary.
   During development, you can use this to watch for regressions:
 
-  $ ./piglit summary html summary/compare results/baseline.results 
results/current.results
+  $ ./piglit summary html summary/compare results/baseline results/current
 
   You can combine as many testruns as you want this way (in theory;
   the HTML layout becomes awkward when the number of testruns increases)
-- 
2.11.0

___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

[Piglit] [PATCH 0/3] Small README clean ups

[Rhys Kidd]

Update or fix a few small items that I noticed when running through piglit
setup on a new development environment.

The first two should be fairly uncontroversial.

The third patch is subjective. It came about when I was running through the
initial suggested smoke tests as-is. It was apparent there was a difference in
the output format for no particular reason midway through. The change is not
carried forward either past those two commands.

This might be a difference without a distinction, but it draws unnececssary
attention away when a user should be learning the principles.

Rhys Kidd (3):
  README: Update the list of test profiles
  README: Correct typo deQP -> dEQP
  README: Make initial smoke tests steps internally consistent

 README | 40 +++-
 1 file changed, 35 insertions(+), 5 deletions(-)

-- 
2.11.0

___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH RFC] framework: Add --skip-missing option

[Dylan Baker]

Quoting Arkadiusz Hiler (2017-07-26 00:46:21)
> Currently, if a test from provided testlist fails to be discovered by
> the framework, piglit blows up with an exception.

Thank you for keeping the default behavior, it's important to have the "run
these and exactly these or fail" option.

> 
> This is both good - for consistency/early errors - and bad - for
> handling some CI/automation scenarios (e.g autobisecting the tests).
> 
> So let's keep the current default, but allow some flexibility with the
> new option that skips the missing tests with a warning.
> 
> bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99649
> Cc: Tomi Sarvela 
> Cc: Petri Latvala 
> Reviewed-by: Martin Peres 
> Signed-off-by: Arkadiusz Hiler 
> ---
>  framework/profile.py  | 5 +
>  framework/programs/run.py | 6 ++
>  2 files changed, 11 insertions(+)
> 
> diff --git a/framework/profile.py b/framework/profile.py
> index a625318..7a39e3c 100644
> --- a/framework/profile.py
> +++ b/framework/profile.py
> @@ -37,6 +37,7 @@ import importlib
>  import itertools
>  import multiprocessing
>  import multiprocessing.dummy
> +import warnings
>  import os
>  import re
>  
> @@ -47,6 +48,7 @@ from framework.dmesg import get_dmesg
>  from framework.log import LogManager
>  from framework.monitoring import Monitoring
>  from framework.test.base import Test
> +from framework.options import OPTIONS
>  
>  __all__ = [
>  'RegexFilter',
> @@ -314,6 +316,9 @@ class TestProfile(object):
>  if self.forced_test_list:
>  opts = collections.OrderedDict()
>  for n in self.forced_test_list:
> +if OPTIONS.skip_missing and n not in self.test_list:
> +warnings.warn("Test {} missing. Skipping.".format(n))
> +continue
>  opts[n] = self.test_list[n]
>  else:
>  opts = self.test_list  # pylint: disable=redefined-variable-type

I agree that we really want this to have an explicit notrun or similar status.
I'm not sure what the right way to do it is. You could probably create some kind
of stub Test instance that just sets a result of NotRun and returns, but that
doesn't feel very elegant.

> diff --git a/framework/programs/run.py b/framework/programs/run.py
> index 6444cfe..bf68f31 100644
> --- a/framework/programs/run.py
> +++ b/framework/programs/run.py
> @@ -208,6 +208,10 @@ def _run_parser(input_):
>   'isolation. This allows, but does not require, '
>   'tests to run multiple tests per process. '
>   'This value can also be set in piglit.conf.')
> +parser.add_argument("--skip-missing",
> +dest="skip_missing",
> +action="store_true",
> +help="skip on missing tests instead of failing")
>  parser.add_argument("test_profile",
>  metavar="",
>  nargs='+',
> @@ -291,6 +295,7 @@ def run(input_):
>  options.OPTIONS.sync = args.sync
>  options.OPTIONS.deqp_mustpass = args.deqp_mustpass
>  options.OPTIONS.process_isolation = args.process_isolation
> +options.OPTIONS.skip_missing = args.skip_missing
>  
>  # Set the platform to pass to waffle
>  options.OPTIONS.env['PIGLIT_PLATFORM'] = args.platform
> @@ -389,6 +394,7 @@ def resume(input_):
>  options.OPTIONS.sync = results.options['sync']
>  options.OPTIONS.deqp_mustpass = results.options['deqp_mustpass']
>  options.OPTIONS.proces_isolation = results.options['process_isolation']
> +options.OPTIONS.skip_missing = results.options['skip_missing']

As the person who added the options module...
Options was a mis-design from the start. I really don't want to see more code
added to options. If there's a way we could pass this some other way I would
very much prefer that.

>  
>  core.get_config(args.config_file)
>  
> -- 
> 2.9.4
> 
> ___
> Piglit mailing list
> Piglit@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/piglit


signature.asc
Description: signature
___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH 1/1] cl: Use required format for sampler test

[Jan Vesely]

ping.

Tom, Zoltan,

you added the original test, does this look OK to you?

thanks,
Jan

On Mon, 2017-07-17 at 17:47 -0400, Jan Vesely wrote:
> CL_RGBA + CL_FLOAT is in minimal supported set.
> Fixes sampler test on beignet (IVB and SKL).
> 
> Signed-off-by: Jan Vesely 
> ---
>  tests/cl/program/execute/sampler.cl | 18 +-
>  1 file changed, 9 insertions(+), 9 deletions(-)
> 
> diff --git a/tests/cl/program/execute/sampler.cl 
> b/tests/cl/program/execute/sampler.cl
> index 45960b778..6de1d4462 100644
> --- a/tests/cl/program/execute/sampler.cl
> +++ b/tests/cl/program/execute/sampler.cl
> @@ -10,12 +10,12 @@ local_size:  1 0 0
>  name: read from image using linear filtering and unnormalized coords
>  kernel_name: readf_f_x
>  
> -arg_in:  1 image float   0.0 1.0 \
> - 2.0 3.0 \
> +arg_in:  1 image float4   0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0   \
> +  2.0 2.0 2.0 1.0 3.0 3.0 3.0 1.0   \
> image_type 2d \
> image_width 2 \
> image_height 2\
> -   image_channel_order INTENSITY \
> +   image_channel_order RGBA \
> image_channel_data_type FLOAT
>  arg_in:  2 sampler normalized_coords 0   \
> addressing_mode NONE  \
> @@ -27,12 +27,12 @@ arg_out: 0 buffer float[4]  0.0  0.5  1.5 
>  1.0
>  name: read from image using linear filtering and normalized coords
>  kernel_name: readf_f_x
>  
> -arg_in:  1 image float   0.0 1.0 \
> - 2.0 3.0 \
> +arg_in:  1 image float4   0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0   \
> +  2.0 2.0 2.0 1.0 3.0 3.0 3.0 1.0   \
> image_type 2d \
> image_width 2 \
> image_height 2\
> -   image_channel_order INTENSITY \
> +   image_channel_order RGBA \
> image_channel_data_type FLOAT
>  arg_in:  2 sampler normalized_coords 1   \
> addressing_mode NONE  \
> @@ -44,12 +44,12 @@ arg_out: 0 buffer float[4]   0.0   0.5   1.5  
>  1.0
>  name: read from image using clamp_to_edge addressing mode
>  kernel_name: readf_f_x
>  
> -arg_in:  1 image float   0.0 1.0 \
> - 2.0 3.0 \
> +arg_in:  1 image float4   0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0   \
> +  2.0 2.0 2.0 1.0 3.0 3.0 3.0 1.0   \
> image_type 2d \
> image_width 2 \
> image_height 2\
> -   image_channel_order INTENSITY \
> +   image_channel_order RGBA \
> image_channel_data_type FLOAT
>  arg_in:  2 sampler normalized_coords 0   \
> addressing_mode CLAMP_TO_EDGE  \


signature.asc
Description: This is a digitally signed message part
___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH] gl30basic: add some extra suspected extension

[Martin Peres]

On 26/07/17 15:58, sandra koroniewska wrote:

Hello,
is this patch ok now? Because it hasn't been pushed.

Regards,
Sandra


It still did not get any Reviewed-by. I guess the summer vacation is 
affecting the piglit team.

___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH] gl30basic: add some extra suspected extension

[sandra koroniewska]

Hello,
is this patch ok now? Because it hasn't been pushed.

Regards,
Sandra

On Thu, Jul 20, 2017 at 12:12 PM, sandra koroniewska <
sandra.koroniew...@gmail.com> wrote:

> Hi,
> if this patch is alright now, can someone push it please?
>
> Regards,
> Sandra
>
> On Fri, Jun 30, 2017 at 4:45 PM, sandra koroniewska <
> sandra.koroniew...@gmail.com> wrote:
>
>> "Is it WGL_EXT_SWAP_CONTROL or WGL_EXT_swap_control? I assume you
>> tested this, so it's the latter (as that's what's in the code)."
>>
>> Yes, it's the latter.
>>
>> On Fri, Jun 30, 2017 at 4:40 PM, Sandra Koroniewska <
>> sandra.koroniew...@gmail.com> wrote:
>>
>>> WGL_EXT_SWAP_CONTROL is added for historical reasons.
>>> ---
>>>  tests/general/gl30basic.c | 12 ++--
>>>  1 file changed, 6 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/tests/general/gl30basic.c b/tests/general/gl30basic.c
>>> index 2207787bb..6e1d903dd 100644
>>> --- a/tests/general/gl30basic.c
>>> +++ b/tests/general/gl30basic.c
>>> @@ -95,12 +95,12 @@ test_extension_list(void)
>>>const GLubyte *ext = glGetStringi(GL_EXTENSIONS, k);
>>>if (0)
>>>   printf("Ext[%d] = %s\n", k, (char *) ext);
>>> -  if (!ext ||
>>> -  ext[0] != 'G' ||
>>> -  ext[1] != 'L' ||
>>> -  ext[2] != '_') {
>>> - printf("%s: bad extension string [%d]: %s\n", Prog, k, ext);
>>> - return PIGLIT_FAIL;
>>> +
>>> +  /* In some drivers WGL_EXT_swap_control is in the extension
>>> +   * string for historical reasons. */
>>> +  if (!ext || (strncmp(ext, "GL_", 3) != 0 && strcmp(ext,
>>> "WGL_EXT_swap_control") != 0)) {
>>> +  printf("%s: bad extension string [%d]: %s\n", Prog, k, ext);
>>> +  return PIGLIT_FAIL;
>>>}
>>>if (strchr((char *) ext, ' ')) {
>>>   printf("%s: extension string [%d] contains a space: %s\n",
>>> Prog, k, ext);
>>> --
>>> 2.11.0.windows.1
>>>
>>>
>>
>
___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

[Piglit] [PATCH RFC] framework: Add --skip-missing option

[Arkadiusz Hiler]

Currently, if a test from provided testlist fails to be discovered by
the framework, piglit blows up with an exception.

This is both good - for consistency/early errors - and bad - for
handling some CI/automation scenarios (e.g autobisecting the tests).

So let's keep the current default, but allow some flexibility with the
new option that skips the missing tests with a warning.

bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99649
Cc: Tomi Sarvela 
Cc: Petri Latvala 
Reviewed-by: Martin Peres 
Signed-off-by: Arkadiusz Hiler 
---
 framework/profile.py  | 5 +
 framework/programs/run.py | 6 ++
 2 files changed, 11 insertions(+)

diff --git a/framework/profile.py b/framework/profile.py
index a625318..7a39e3c 100644
--- a/framework/profile.py
+++ b/framework/profile.py
@@ -37,6 +37,7 @@ import importlib
 import itertools
 import multiprocessing
 import multiprocessing.dummy
+import warnings
 import os
 import re
 
@@ -47,6 +48,7 @@ from framework.dmesg import get_dmesg
 from framework.log import LogManager
 from framework.monitoring import Monitoring
 from framework.test.base import Test
+from framework.options import OPTIONS
 
 __all__ = [
 'RegexFilter',
@@ -314,6 +316,9 @@ class TestProfile(object):
 if self.forced_test_list:
 opts = collections.OrderedDict()
 for n in self.forced_test_list:
+if OPTIONS.skip_missing and n not in self.test_list:
+warnings.warn("Test {} missing. Skipping.".format(n))
+continue
 opts[n] = self.test_list[n]
 else:
 opts = self.test_list  # pylint: disable=redefined-variable-type
diff --git a/framework/programs/run.py b/framework/programs/run.py
index 6444cfe..bf68f31 100644
--- a/framework/programs/run.py
+++ b/framework/programs/run.py
@@ -208,6 +208,10 @@ def _run_parser(input_):
  'isolation. This allows, but does not require, '
  'tests to run multiple tests per process. '
  'This value can also be set in piglit.conf.')
+parser.add_argument("--skip-missing",
+dest="skip_missing",
+action="store_true",
+help="skip on missing tests instead of failing")
 parser.add_argument("test_profile",
 metavar="",
 nargs='+',
@@ -291,6 +295,7 @@ def run(input_):
 options.OPTIONS.sync = args.sync
 options.OPTIONS.deqp_mustpass = args.deqp_mustpass
 options.OPTIONS.process_isolation = args.process_isolation
+options.OPTIONS.skip_missing = args.skip_missing
 
 # Set the platform to pass to waffle
 options.OPTIONS.env['PIGLIT_PLATFORM'] = args.platform
@@ -389,6 +394,7 @@ def resume(input_):
 options.OPTIONS.sync = results.options['sync']
 options.OPTIONS.deqp_mustpass = results.options['deqp_mustpass']
 options.OPTIONS.proces_isolation = results.options['process_isolation']
+options.OPTIONS.skip_missing = results.options['skip_missing']
 
 core.get_config(args.config_file)
 
-- 
2.9.4

___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

[Piglit] [PATCH RFC] Handling missing tests

[Arkadiusz Hiler]

Hey all,

This is my initial (and simple) attempt at adding a way to handle missing tests
gracefully for an automated CI/bisecting/etc. scenarios.


The change is okay(-ish) on it's own but we can make the missing tests case more
of an first-class citizen, e.g.:

 1) actually report the missing tests as notrun, instead of ignoring them with
a warning

 2) add another possible result state - missing, so it would be more obvious

But those would require much more work and code juggling, so I would like to
know your opinions first.

Any thoughts?


Arkadiusz Hiler (1):
  framework: Add --skip-missing option

 framework/profile.py  | 5 +
 framework/programs/run.py | 6 ++
 2 files changed, 11 insertions(+)

-- 
2.9.4

___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit