Il 16/05/2014 23:28, Ademar Reis ha scritto:
The leaves are:
env->production
env->debug
tests->sync_test->standard
tests->sync_test->aggressive
This will get multiplexed into 4 test scenarios, thus requiring 4
test runs:
env->production + tests->sync_test->standard
env->production + tests->sync_test->aggressive
env->debug + tests->sync_test->standard
env->debug + tests->sync_test->aggressive
It's easy to notice that the system grows exponentially with the
number of variants.
So far so good. What would it be without the filter-out: "tests"?
## default values ##
The multiplex system is optional. Tests are supposed to run with
default values in a supported system, just like virt-test does
today. Test writers declare the defaults (not part of the
multiplexing system).
I'm not sure I understand the ramifications of this.
Machine types should be part of the multiplexing system, so that the
user can run tests with many machine types.
However, (some) defaults should come from the machine types, not from
the user, because different machine types have different defaults. Such
defaults should be picked up automatically unless the user overrides
them. A user should be able to just ask "run this test on Linux/x86
Windows and Linux/Cubieboard" and get virtio with Linux/x86 guests, IDE
with Windows guests, and an SD card with the Linux/Cubieboard guests.
If the user says "default = IDE", the test framework should run the
tests with IDE disks on all guests. If the user says "default = SCSI",
the test framework should cut out the Cubieboard and only run tests
(with SCSI) on Linux/x86 and Windows.
I hope that by enforcing structure and naming of variants, we can do
this without the shortcomings of Cartesian config.
Paolo
So when a branch is filtered out, the test will fallback to the
defaults provided by the test runner. Ditto for when tests are
run without the multiplexer.
_______________________________________________
Virt-test-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virt-test-devel
