On 03.05.2016 00:21, Dylan Baker wrote:
This series makes the test ordering in piglit deterministic, in that it will run the tests in the order that they are provided by the profile. For native piglit tests (OpenGL and OpenCL) this will make them completely deterministic, for external integrated suits like dEQP and IGT it will be determined by the way the external suite enumerates it's tests; if the external tools is deterministic piglit will be too. An option for external suites that are not deterministic would be to implement a custom TestProfile class that sorts test TestDict using it's reorder() method. This further alters --test-list option to work with this internal change to impose ordering restrictions on the way tests are run. Brian, I hope this is what you're looking for. Daniela, I don't know exactly what you were looking for when you asked about sort order, but I thought I would point you at this, since Brian asked for something (that I think) is similar to what you're looking for. (https://bugs.freedesktop.org/show_bug.cgi?id=95053)
Yes, this is what we (i-g-t) were looking for. Thank you! Tested-by: Gabriel Feceoru [email protected]
Dylan Baker (5): framework/profile: Convert TestDict to a collections.MutableMapping framework/profile: don't replace TestDict instance with regular dict framework/profile: Make test run order deterministic framework: Make JSON backend test ordering deterministic framework: Make testlist option impose ordering framework/backends/json.py | 9 ++--- framework/profile.py | 84 +++++++++++++++++++++++++++++++++++----------- framework/programs/run.py | 12 +++---- framework/results.py | 2 +- unittests/profile_tests.py | 55 +++++++++++++++++++----------- 5 files changed, 112 insertions(+), 50 deletions(-)
_______________________________________________ Piglit mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/piglit
