On Fri, May 16, 2014 at 06:50:47PM +0200, Paolo Bonzini wrote:
> Il 16/05/2014 17:42, Ademar Reis ha scritto:
> >I believe this lack of structure is the root cause of many of the
> >problems or dislikes we have with the system.
>
> I agree.
>
> >It might be better to explain the concept with an hypothetic
> >example:
> >
> >## reference tree ##
> >
> > * virtio_net
> > - filter: "only guest->os->linux" # meaning: only
> > # the linux leave
> > # of guest->os will
> > # be used
>
> Interesting. Why a string that has to be parsed, and not something like
>
> - filter-only: "guest->os->linux"
>
Sure.
> or even (similarly, "conflicts" for Cartesian config "no")?
>
> - requires: "guest->os->linux"
Hmm, I'm not a big fan of "only", but it's stronger (more
restrictive) than "requires" because it reflects the fact that a
filter-only operation "removes all sibling branches from the
tree".
For completeness, I should have added this to the RFC:
## filter behavior and tree multiplexing ##
Filters chop the tree, removing branches. After filters are
applied, whatever leaves are left get multiplexed.
There are two filter types:
- filter-only <branch>: removes all sibling branches from the
tree. For example, "filter-only: guest->os->linux" will remove
everything else under "guest->os", leaving only linux.
- filter-out <branch> ("no" in the original draft): remove this
particular branch from the tree, but leave everything else in.
For example, "filter-out: env" will remove the entire env
branch from the tree.
For example, suppose that after applying several filters I'm left
with the tree below:
<ROOT>
* env
* production
- malloc_perturb = no
- gcc_flags = -O3
* debug
- malloc_pertub = yes
- gcc_flags = -g
* tests
* sync_test
* standard
- sync_timeout = 30
- sync_tries = 10
* aggressive
- sync_timeout = 10
- sync_tries = 20
The leaves are:
env->production
env->debug
tests->sync_test->standard
tests->sync_test->aggressive
This will get multiplexed into 4 test scenarios, thus
requiring 4 test runs:
env->production + tests->sync_test->standard
env->production + tests->sync_test->aggressive
env->debug + tests->sync_test->standard
env->debug + tests->sync_test->aggressive
>
> > - nic_model = virtio
> > - enable_msix_vectors = yes
>
> > * tests
> > - filter: "no tests" # we don't want to multiplex test
> > # variables from different tests, so we
> > # filter out the entire branch
>
> I don't get this. :)
"tests" is a branch of the tree, but we don't want tests mixed
together.
If we don't use such filter, when running a sync_test we would
end up with a matrix with entries like:
tests->sync_test->standard... + tests->ping_test->standard...
tests->sync_test->standard... + tests->ping_test->aggressive...
tests->sync_test->aggressive... + tests->ping_test->standard...
tests->sync_test->aggressive... + tests->ping_test->aggressive...
I wanted the system to work on a single tree for simplicity.
>
> > * sync_test
> > - filter: "only guest->os->linux" # we don't want to
> > # multiplex other OSes here
> > - filter: "only hardware->disks" # self-explanatory
>
> Do you mean that leaves other than hardware->disks will get the default
> values from say hardware->network? So it would be, more precisely:
>
> * network
> - nic_model = none
> * rtl_8139
> - nic_model = rtl8139
> * e1000
> - nic_model = e1000
>
> which seems to work for network.
>
> But then you have a problem for disks, where "none" is not really a
> possibility. Here you have:
I see. I should have explained how the defaults system is
supposed to work.
The multiplex system is optional. Tests are supposed to run with
default values in a supported system, just like virt-test do
today. Test writers declare the defaults.
So when a branch is filtered out, the test will fallback to the
defaults. Ditto for when tests are run without the multiplexer.
>
> > * ping_test
> > - filter: "only guest->os->linux" # we don't want to
> > # multiplex other OSes here
> > - filter: "only hardware->network" # self-explanatory
>
> This would mean that hardware->disks would not be expanded. You would have:
>
> * hardware
> * disks
> - drive_format = ?!? # some machines have default ide,
> # others have virtio... how to
> # express this?
> * ide
> - drive_format = ide
> * scsi
> - drive_format = scsi
> * virtio
> - drive_format = virtio
In other words, for the ping_test I'm declaring I don't care what
the disk configuration is because as a test writer I'm assuming
only the "hardware->network" variants will impact my test
results.
>
> But then how can an ARM variant say what the default is? If you do this:
>
> * arm
> * virt
> - filter: "only hardware->disks->virtio"
>
> It would conflict with the "only hardware->disks" in the tests.
Filters are combined, so I don't see the conflict. "only
hardware->disks" is stronger than "only hardware->disks->virtio".
The end result will be a tree without any disk variants.
>
> Perhaps this could work:
>
> * arm
> - default: { key: "hardware->disks", value: "virtio" }
> - default: { key: "hardware->network", value: "virtio" }
>
> ...
>
> * ping_test
> - filter: "only guest->os->linux" # requires suggested :)
> - expand: "hardware/network"
>
> So by default neither hardware/disks nor hardware/network are expanded,
> because there is a "default" clause somewhere. However, tests can choose to
> expand other branches. The user can also choose to expand a branch.
>
> Conflicting defaults are an error. Perhaps user defaults could be allowed
> to override system defaults, though...
The test writer declares the defaults and they are the last
fallback.
I expect to expand the tree that is left after filtering, so I
don't get the motivation for "expand".
>
> >## multiple files, namespaces ##
> >
> >We have one single tree, but we can use multiple files to declare
> >the variants.
>
> This is brilliant. The current directory system could be allowed too.
Yep.
>
> > * synctest.cfg:
> > """
> > filter: "no env->host"
> > filter: "no env->guest"
> > """
They should be "no host" and "no guest", my mistake.
> >
> > * $ avocado --multiplex=file.mplx synctest
> > * --> using the config file above, this will multiplex only
> > the env variables, resulting in two variants: production
> > and debug.
>
> This is similar to what you wrote as "only hardware->disks". I
> think there is some confusion here.
In the example above I'm assuming we're using the filters from
the synctest.cfg file, so "only guest->hardware->disks" from the
tree is overridden by "no guest" from the config file.
>
> Would you try rewriting your example in terms of
> expand/default/requires/conflicts and see if it works out right?
>
Sure, I think there's plenty of room to work on the filtering
system, assuming the idea of representing variants in a tree and
chopping it with filters is solid.
Thanks for the feedback!
- Ademar
--
Ademar de Souza Reis Jr.
Red Hat
^[:wq!
_______________________________________________
Virt-test-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virt-test-devel
