On Tue, Jul 19, 2016 at 04:33:02PM +0200, Lukáš Doktor wrote: > ... > > > > To summarize it here (once again): > > > > > > 1. The tree is a "database" which allows injecting values under certain > > > keys > > > with given path (namespace would be also acceptable by me, I don't like > > > the > > > tag, though). So far we agreed that we should turn the yaml=>tree > > > pseudoplugin to real plugin and we are looking for a name (I'd call it > > > yaml2params). > > > 2. Mux is the variant generation mechanism, which is responsible for > > > walking > > > the database and producing variants (which are lists of path along with > > > their values) > > > 3. AvocadoParams is a per-variant database used inside tests to query for > > > values > > > > > > So (1) is actually a core API (independent on yaml parser) which feeds the > > > Mux, the (2) is used by the runner (or job API?) to produce multiple > > > variants of the same test and the (3) is the API available inside test to > > > query for the current test parameters. > > > > > > The (1), (2) and (3) are essential parts of avocado, independent on yaml > > > and > > > currently should stay in `core`, even though that in the future we might > > > want to allow replacing the (1) and maybe even (2) with plugins. Currently > > > this is not important and we are not aware of anyone demanding it. We take > > > (3) as definite for now, so it should not change apart from optimizations. > > > > I think we have different views of what the abstraction layer > > should be. But I don't think this is going to be used anytime > > soon by 3rd parties and we can always make more changes in the > > future, so it's OK to disagree here. > Well I'm not saying this is the final stage, it's the bare minimum for > removing yaml-dependent part from core and allow custom plugins as params > sources. > > > > > > > > > The workflow is: > > > > > > 1. params-feeder (eg. `yaml2params`, or `--mux-inject`) -> inject records > > > into `tree` (in the future it could be any arbitrary database, currently > > > hardcoded as tree) > > > > I think we need a better vocabulary. In particular, we need to > > define what params are. Are params branches/leaves of the tree? > > Or are params the result of the processing of a tree? > > > AvocadoParams is a driver to query data from the current variant. The > current variant is a product of Mux which is currently defined as list of > TreeNodes (as a consequence this currently means leaves, but one can > manually supply any TreeNode object). > > > > 2. `tree` merges all records and applies filters > > > > Filters are part of the tree structure? So are the multiplex > > flags? Again, if this is exposed at the avocado-core API level, > > we're exposing too many implementation details and using the > > (current) multiplex as the API, instead of something more generic > > and abstract. That's why I think the core API should be about > > params (as defined by me previously), not about trees to be > > processed. > Filters are applied on trees. They are applied by Mux object when the tree > is final (my sentence before was a bit misleading, sorry about that). Think > about trees as about special database which contain: > > * path > * environment (key+value pairs) > * tags (!mux, !filters, ...) >
That was my understanding, so I think I got it right. The "tree structure" I mentioned is what you're calling a "special database". I don't like the complexity of it, I think these are mux implementation details which should not be in the API, which should be way, way more generic and abstract. But I'm repeating myself and I think we're lost with too many low level details while the high level architecture is still unclear. You could send a v2 of this RFC, or move on and submit a PR. As long as these APIs are internal (core, therefore without any promise of API stability), we should be fine. Thanks. - Ademar > > > > > > 3. `Mux` object is created on top of the tree > > > 4. test runner uses `Mux` to produce variants and runs tests+variant > > > combinations > > > 5. each test creates AvocadoParams from the variant > > > > > > Currently it works a bit differently (inherited from the old days) because > > > the `yaml2param`, currently `core.tree.create_from_yaml`, actually creates > > > the tree database directly. This is not a problem now, it would be no > > > problem even after turning it into a proper plugin, but it'd become a > > > problem if we chose to replace `tree` with any arbitrary database, > > > therefor > > > I think it's time to extract the `create_from_yaml` and instead of > > > creating > > > the tree we should only ask the `tree` (or database) to > > > inject/update/remove > > > variables/filters/flags. Then we have a clear interface. > > > > > > So the multiplexer is core and IMO should stay there. Only the > > > params-feeders should be extracted, which means the PyYAML would be > > > optional. Also multiplexer is just a database with combinations so it > > > could > > > be used for anything, also we want to keep using one database for test > > > variants and not mixed settings/config_files/test_params. > > > > > > I hope everything is clear, because if not I'd have to draw a > > > chart :-D > > > > I think I get what you're proposing. I would do things > > differently, but this is all going to be internal for a while > > more and we can improve it incrementally. > Sure, this is the first (actually second, the first one was to split > tree+multiplexer and create the Mux object to define the avocado->variant > interface) step. How I see params now is: > > 1. params-feeders => --mux-inject or yaml parser which should inject > path/environment/tags into the tree (or let's call it a database, should > become an independent plugin in the close future) > 2. tree (database) of path/environment/tags. It's purpose is to store > key/value pairs as well as information which allows Mux to produce different > variants (basically slices of the database's environment) > 3. Mux which takes the tree (database), applies filters and allows producing > variants > 4. variant => list of path/environment mappings (slice of the tree), > currently list of full TreeNode structures. Only their path+environment are > used (so if you insist we can only send tuple(path, key+values)) > 5. AvocadoParams => driver to get values based on key+path from the variant > > Lukáš > > > > > Thanks. > > - Ademar > > > > > > > > Regards, > > > Lukáš > > > > > > > Thanks. > > > > - Ademar > > > > > > > > > 2. multiplexer - to produce variants > > > > > 3. avocado params - to obtain params from variant > > > > > > > > > > We should probably find a suitable name to ambiguously identify each > > > > > part as > > > > > of today we only have multiplexer and avocado params, which can lead > > > > > to > > > > > confusions. I can't come up with any good name so unless you have a > > > > > good > > > > > name we maybe end up with the same name. The situation should be > > > > > better, > > > > > though, because those parts will be really separated. > > > > > > > > > > Thank you for the feedback, let's get my hands dirty. :-) > > > > > > > > > > Regards, > > > > > Lukáš > > > > > > > > > > > > > > > > > > Regards, > > > > > > > Lukáš > > > > > > > > > > > > > > > > > > > [1] - https://docs.python.org/2.6/glossary.html#term-iterable > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Ademar Reis Red Hat ^[:wq! _______________________________________________ Avocado-devel mailing list [email protected] https://www.redhat.com/mailman/listinfo/avocado-devel
