Dne 27.11.2014 v 14:06 Paolo Bonzini napsal(a):
On 26/11/2014 20:56, Lukáš Doktor wrote:
This looks amazing and should be doable in YAML. The `mux-default`
should not be needed. Let me explain the idea here. Do you remember the
ways to execute test from my email:
1) default variant
2) full test
3) all related
4) multiplex all
And the places where variants are stored:
A) yaml file added on cmdline (/hw, /os, ...)
B) each test's associated file (/tests/$name/duration,
/test/$name/function, ...)
Z) for default variables (new idea), let's say
`avocado.yaml` which would be 1 level yaml with all default variables
(image: "Fedora", passwd: "123456", ...)
When you execute test as 1), it'd take default values from Z) and than
get only first level values from B)
In case you execute 2), it'd again get default values from Z) and then
it'd parse the B) tree.
The 3) would need to parse Z), A) and B) and multiplex only mux-choices
specified in the B).
The 4) would parse Z), A) and B) and multiplex everything what is not
filtered out.
In my case you'd have:
1 and 2) command line switch --mux=none or --no-mux: take default values
from B and Z, apply mux-expands found in A (but ignore mux-expands
specified in B), filter. The difference is simply that in case 1 you
have no mux-expands in A (a mux-expand can also be given on the command
line, BTW).
3) command line switch --mux=auto or just --mux: take default values
from B and Z, expand mux-choices specified in A and B via mux-expand
4) command line switch --mux=all: take default values from B and Z,
apply mux-expand to all choices, filter out.
Yep, the possibility to multiplex on cmdline was intended :-)
######################################################################
Lucas also pointed out, that I didn't explain the hooks sufficiently. So
how the params.get() should look like:
def get(key, default):
raw_value = super(Params, self).get(key, default)
if key in self.hooks:
return self.hooks[key](raw_value, default)
else:
return raw_value
where hook could eg. means:
def arch_hook(raw_value, default):
# params.get("/hw/arch")
if not raw_value:
if default:
return default
else:
return arch.get_supported_archs()[0]
elif raw_value not in arch.get_supported_archs():
if default:
return default
else:
raise TestNAError("Architecture ... not supported")
I think actually it should always raise TestNAError in this second case,
giving the following very simple code for arch_hook:
value = raw_value or default or arch.get_supported_archs()[0]
if value not in arch.get_supported_archs():
raise TestNAError("Architecture ... not supported")
return value
However, I think it is important to have a declarative way to describe
the muxing and the defaults. A declarative syntax makes it possible to
convert muxing constraints to Boolean formulas, and hence enumerate the
results with binary decision diagrams (BDDs). In turn, this drops all
the complicated optimization logic of the cartesian config parser.
Now that I thought about it more, I recalled another idea from the old
thread, which is automatic expansion of filtered variants. In this idea
you'd have two variants of mux-only and mux-no:
- mux-requires-only and mux-requires-no, mostly for use in B and Z, do
nothing special
- mux-only and mux-no, mostly for use in A, automatically expand the
choices that are mentioned. In other words,
- !mux-only { '/os': 'linux' }
is the same as
- !mux-expand '/os'
- !mux-requires-only { '/os': 'linux' }
The idea is that in A you'd say
- !mux-no { '/hw/net/backend': 'vhost' }
meaning you want to test all backends except vhost. In B you'd say instead
- !mux-requires-no { '/hw/net/backend': 'user' }
meaning you want to skip this test, but not override the choice of the
user. It's also possible to do this magically (i.e. mux-only in A
always implies mux-expand) and avoid proliferation of tags. Still, the
difference in behavior should probably be taken into account.
Does this let you build arbitrary DNF (disjunctive normal form)
formulas? You can do ANDs of terms like
P | Q | R ~(P | Q | R)
You can only have direct and negated variables in the same formula if
you replicate the mux-group/mux-choice structure in A; this is quite
unwieldy. In the cartesian config this is provided by:
Linux:
only virtio
Windows:
only rtl8139
where Linux and Windows will match anywhere in the output tuple. What
we can do then is add a !mux-if tag:
- !mux-if { '/os': 'linux' }
- !mux-only { '/hw/net/nic_model': 'virtio' }
- !mux-if { '/os': 'windows' }
- !mux-only { '/hw/net/nic_model': 'rtl8139' }
This translates to the following Boolean formula
(os == linux => hw.net.nic_model == virtio) &&
(os == windows => hw.net.nic_model == rtl8139)
where I've written '/os/distro' more easily as 'os.distro' (probably
it's a good idea to write it that way in mux-{if,only,no} as well).
Rewriting the implication A => B to ~A | B:
(os != linux || hw.net.nic_model == virtio) &&
(os != windows || hw.net.nic_model == rtl8139)
and adding conditions to govern expansion of the choices:
(os != linux || hw.net.nic_model == virtio) &&
(os != windows || hw.net.nic_model == rtl8139) &&
expand(hw.net.nic_model)
Note that os only appears under a mux-if, so it is *not* expanded.
So, how are the mux directives translated to SAT formulas? Pretty
easily, I might say.
A choice with N possibilities is mapped to N+1 SAT variables, N for the
possibilities and 1 for "should this choice be expanded". So in the
case above there are six SAT variables:
expand(os)
os == linux
os == windows
expand(hw.net.nic_model)
hw.net.nic_model == virtio
hw.net.nic_model == rtl8139
Let's first define a "prefix" variable. It represents the !mux-if and
!mux-choice above the currently parsed element of the tree. So in the
case above, the prefix of the first !mux-if is "os == linux".
Similarly, when defining
- !mux-choice os:
- linux:
- !mux-choice distro:
- fedora:
- !mux-choice version:
- 13:
- 14:
- debian:
- !mux-choice version:
- squeeze:
- jessie:
- rhel:
the prefix while parsing fedora versions is "os == linux && os.distro ==
fedora"; the prefix while parsing debian versions is "os == linux &&
os.distro == debian"; and so on.
We'll use the negated prefix often, in order to build ORs instead of
implications. It is derived simply with De Morgan laws. The negation of
"os == linux && os.distro == fedora"
is
"os != linux || os.distro != fedora"
i.e. in terms of the variables we have defined above:
"!(os == linux) || !(os.distro == fedora)".
The negated prefix will be written prefix'.
With this in mind, here is how you do the conversion to SAT.
- !mux-choice name
A:
- mux-default: True
...
B:
...
C:
...
Your variables are
path.name == A
path.name == B
path.name == C
expand(path.name)
!expand(path.name)
which I'll write shortly as pA, pB, pC, pE, pE'. You first need
N(N-1)/2 terms to express mutual exclusion, and N-1 terms to express the
defaults. On the left they're written with implications, on the right
with disjunctions:
pA => ~pB ::: ~pA | ~pB (mutual exclusion)
pA => ~pC ::: ~pA | ~pC
pB => ~pC ::: ~pB | ~pC
~pE => ~pB ::: pE | ~pB (default)
~pE => ~pC ::: pE | ~pC
In addition, as mentioned above, the prefix is modified during the
parse. "& pA" is added to the prefix (and "| ~pA" to the negated
prefix) while parsing under "- A:", and so on.
- !mux-expand name
This is a simple term (left = implication, right = disjunction):
prefix => expand(path.name) ::: prefix' | expand(path.name)
This forces pE to be true and pE' to be false.
- !mux-if X
During the parse, X is added to the prefix, and ~X is added to the
negated prefix. No terms are emitted.
- !mux-requires-only { 'X': 'Y', 'U': 'V' }
Let's write the "X==Y" variable as pXY and the "U==V" variable as pUV.
You have the following termv (again, left uses arbitrary Boolean
formulas while right uses disjunctions only):
prefix => pXY | pUV ::: prefix' | pXY | pUV
- !mux-requires-no { 'X': 'Y', 'U': 'V' }
In this case every member of the map produces a separate term:
prefix => !pXY ::: prefix' | !pXY
prefix => !pUV ::: prefix' | !pUV
In order to produce the result of the multiplexing, you unfortunately
cannot just find all solutions to the SAT problem. This is because a
solution with pE = true will always be present; all the above does is
enforce no solution exists with pE = false when a choice is expanded.
There is thus a lot of redundancy in the solution computed so far.
Luckily, the BDD will represent cheaply the redundant solution _and_
will provide a way to easily cull the invalid solutions. To do so, we
can perform the following steps:
- ensure that, in the BDD, the expand(foo) variable lies above the
foo==X variables
- when visiting the BDD, if the 0 branch of expand(foo) has a solution,
skip visiting the 1 branch altogether.
Paolo
Oh my... Thank you for this elaborate. I'm going to need a board and
couple of hours/days to unparse and understand it.
Anyway the original idea behind the these hooks was not to make choices,
or cut down the tree. Original idea was just to create full tree (or
just defaults), execute the test and let it fail in case the dependency
is not met. Your solution (if I understand it correctly) goes further
and removes these tests from multiplexation. On the other hand, I don't
yet understand, how it works :-D
My approach was to execute the hooks in case user/test/framework asks
for the param while executing the test. Therefor the `None` architecture
makes sense, because it's just not set yet and it'll automatically
choose the right one. The same way would work the network for windows
vs. linux guests. You won't specify the default, but once the
framework/test asks for params.get('/virt/hw/nic_model') it'd execute hook:
def nic_hook(raw_value, default):
if raw_value is None:
return net.get_supported_nics()[0]
elif raw_value in net.get_supported_nics():
return raw_value
elif default:
return default
else:
raise TestNAError("Unable to match nic_model...")
Where the nic.get_supported_nics() would use params["/virt/os/distro"].
The benefit is, that on complex trees you'd not need to check all hooks,
only the ones which applies for given test. The cons against your idea
(if I understood it correctly) is, that your version would remove
non-suitable variants during multiplexation, my version would execute
the test up to the point where it gets to the params.get(trouble_maker)
call (where it would generate the TestNAError).
So correct me if I'm wrong, but my impression is that for bigger trees
with lots of irrelevant hooks your variant can be actually slower. For
small trees without variants, they should be probably the same speed and
for big trees with relevant hooks yours should be faster (and generates
smaller report as it'd skip irrelevant tests).
Anyway please give me couple of days to actually learn about the magic
you used, than I might completely change my opinion. In this email I
just wanted to clarify my original thought...
Regards,
Lukáš
Similarly you can handle -device support, default drive format, ...
Anyway I'll take a deeper look at the mux domains you proposed, it's
named version of what I had in mind (and better supported in yaml).
Thank you for your response,
Lukáš
I'm not 100% sure this is valid YAML, but should be close.
Thanks,
Paolo
_______________________________________________
Virt-test-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virt-test-devel
_______________________________________________
Virt-test-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virt-test-devel
_______________________________________________
Virt-test-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virt-test-devel
