On 04/29/2014 08:48 AM, Plamen Dimitrov wrote:
Hi guys,
Since all possible changes in the Cartesian configuration format concern
our tests quite much, I want to share my opinion on some of the point
below.
On 04/29/2014 04:55 AM, Lucas Meneghel Rodrigues wrote:
From the feedback gathered so far, the work items are:
1) Complexity
Solution: Implement Paolo's RFC outlined here:
https://www.redhat.com/archives/virt-test-devel/2014-February/msg00073.html
Would be a start
Paolo's proposal is good in the fact that it is an extension of the
currently available functionality. My main concern here is losing the
original meaning of "@" which I find rather useful. I think you will all
agree with me that short names instead of lengthy line-breaking strings
improve readability where desired. Can we perhaps keep the original
meaning of "@" at use another non-reserved character for this purpose?
It would be better than changing its meaning and finding another
character for name shortening for sure since people (and scripts) are
already accustomed to the current meaning. In this way we extend without
harming what we already have which is by no means useless. My second
point here would be that this is not really reduction of complexity,
since the general format is preserved and the only thing we do is even
add more complexity, however I guess for the better good.
We can find out another marker.
2) Frailty
Solution: Make the parser able to tell people wether they have messed
up their indentation, and be loud and fail instead of silently
ignoring the non-perfectly-aligned data.
I completely agree with this.
3) Inflexibility
Solution: Create a method that expands what the generators created,
and order according to the order the user asked for.
In our custom test suite we already have some considerable advances
here. It would be my pleasure to propose our solution for reviewing by
attaching two documents here or in separate e-mail. Of course only if
you wish so.
Sure, we'd love to see them. Either here or in a separate e-mail seems fine.
4) Generate variants from tests
Solution: Well, no idea on this one yet :)
I would generally deprecate such practice as the complexity will be
raised even more. In our experience and developed tests we haven't
encountered a need for some thing like this but I guess this is why I
cannot really prove a point here.
Eduardo mentioned one example where said feature would be useful: In his
cpu id tests he has currently two options:
1) Write a single test that iterates through all CPU models on the fly
(that is, we query qemu for all the models and then iterate through the
list of models)
2) Write a cartesian file with all CPU models 'hard coded', then have a
much better test granularity, at the expense of manually maintaining the
file
It is very much a question of test granularity. In general, smaller
granularity leads to identifying trouble spots more easily, the review
time of failures being smaller, since each individual test is simpler.
Let us know whether you agree with my assessment, and if there's
something I missed, including a better format/way to solve the
original problem. If you read so far, thank you very much for your
patience!
I hope you will take into account my concerns and moreover enjoy our
extension. After all, Cartesian configuration is my favorite part of
virt-test :)
This statement by itself is already very helpful. It's another positive
experience with the format, indicating that it might be a decent
solution for the problem at hand. Thanks!
_______________________________________________
Virt-test-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virt-test-devel
