On 05/02/2016 03:21 PM, Dylan Baker wrote:
This series makes the test ordering in piglit deterministic, in that it
will run the tests in the order that they are provided by the profile.
For native piglit tests (OpenGL and OpenCL) this will make them
completely deterministic, for external integrated suits like dEQP and
IGT it will be determined by the way the external suite enumerates it's
tests; if the external tools is deterministic piglit will be too.
An option for external suites that are not deterministic would be to
implement a custom TestProfile class that sorts test TestDict using it's
reorder() method.
This further alters --test-list option to work with this internal change
to impose ordering restrictions on the way tests are run.
Brian, I hope this is what you're looking for.
Yes, it is. Thanks!
I applied the patchs and tried a few test runs and it looks good.
I just did a light review of the code changes.
Acked-by: Brian Paul <[email protected]>
Tested-by: Brian Paul <[email protected]>
One other issue though. It would be nice if the output of
piglit-print-commands.py could be used as a test list input to
piglit-run.py When I tried it, piglit-run.py spun its wheels for a
while then hit an exception.
-Brian
Daniela, I don't know exactly what you were looking for when you asked
about sort order, but I thought I would point you at this, since Brian
asked for something (that I think) is similar to what you're looking
for.
(https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.freedesktop.org_show-5Fbug.cgi-3Fid-3D95053&d=BQIBAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=T0t4QG7chq2ZwJo6wilkFznRSFy-8uDKartPGbomVj8&m=P0idU-rSN_1ROsAgtRw0-jqXAQvrw1lv886Sa7DT818&s=KhSdC9bxSuoRx9psA_FmzE8UIKCcTtyWCzT7zBXOQ08&e=
)
Dylan Baker (5):
framework/profile: Convert TestDict to a collections.MutableMapping
framework/profile: don't replace TestDict instance with regular dict
framework/profile: Make test run order deterministic
framework: Make JSON backend test ordering deterministic
framework: Make testlist option impose ordering
framework/backends/json.py | 9 ++---
framework/profile.py | 84 +++++++++++++++++++++++++++++++++++-----------
framework/programs/run.py | 12 +++----
framework/results.py | 2 +-
unittests/profile_tests.py | 55 +++++++++++++++++++-----------
5 files changed, 112 insertions(+), 50 deletions(-)
_______________________________________________
Piglit mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/piglit
