On Mon, 8 Feb 2016 06:54:17 -0500 (EST) Josef Skladanka <[email protected]> wrote:
> This is an initial take on stuff that was discussed in person during > Tim's stay in Brno. Sending to the list for additional > discussion/fine-tuning. > = What = > > Talking rpmgrill-like checks, there will be a need to be able to > facilitate some kind of structure for representing that a check is > composed of multiple subchecks, for example: > > check - FAILED > subcheck1 - PASSED > subcheck2 - PASSED > subcheck3 - FAILED > subcheck4 - PASSED > > !IMPORTANT: ResultsDB will not be responsible for computing the > result value for an "upper level" Result from the subchecks - this is > the check's (check developer's) responsibility. Are we going to require an overall status for the grouped subchecks? Would ResultsDB do any of the grouping outside of facilitating the namespaces? > This could (should?) be done on two levels: > * physicall nesting the Results as such in the database structure > * namespacing Testcases > > For the start, we decided to go with the simplistic approach of > nesting the Testcases via a simple namespacing - thus allowing a > frontend/query tool to reconstruct the structure at least to some > extent e.g. by relying on a premise, that Results that are a part of > one Job can be converted to a tree-like structure, based on the > Testcase namespacing, at least to some extent, if needed. > > > == Namespace structure == > > We'll be providing some top-level namespaces (list not yet final): > * app > * fedoraqa > * package > * scratch (?) > > These will the further split to facilitate for a finer level of > granularity, e.g.: > > app > testdays > powermanagement > pm-suspendr > fedoraqa > depcheck > rpmgrill > package > <pkgname> > unit > func > > Everything below the top-level will be 100% user defined. We might > have recommendations for specific namespaces (like > package.<pkgname>), but we won't be enforcing them. > > The structure will be implemented (at least in the initial > implementation) just via the Testcase.name attribute in the DB, using > dots as a separator. Later on, we can easily add an easy way of using > wildcards for searching (e.g. app.testdays.*.pm-suspendr) > > !IMPORTANT: the namespaces are not to be used to represent > "additional data" about the underlying result such as architecture, > item under test, etc. This is what the Result's extra-data > (ResultData) is there for. > > NOTE: Although we do not encourage to store the results to the finest > granularity "just because" (e.g. individual results of a unittest > testsuite), we leave it to the check-developer's judgement. If there > is a usecase for it, let them do it, we don't care, as long as the DB > is not extremely overloaded. To nitpick a bit, I had to read the first part of that a couple times before understanding. Maybe something like: Although the check developer has the final say over the granularity of results stored, we do not suggest storing results simply for the sake of having them That being said, how many results are we talking about here if the DB would get overloaded? > == Authentication/Authorization == > > We'll be continuing with the "expect no malice" approach we have > right now. There will be just a simple limitation in libtaskotron: > > check git clone > if cloned: only allow non-pkg namespace if __our__ repo > else: do whatever, don't care > > in libtaskotron: > check the git checkout like listed above > have whitelisted napespace repos in config > > !FIXME: the mechanism above is just copied from tflink's notes, I > can't remember the details :/ My memory of the idea is: The check that we'd do for non-pkg namespaces would be to look at the git checkout during task execution - if that repo was cloned from a set of whitelisted repos (kept in config), it would be allowed to use non-pkg namespaces when reporting to resultsdb. It's certainly not perfect and could be worked around without too much trouble but I think it'll be enough so long as we have sane behavior in the case where we don't even find a repo to look at. Putting the repo-checking code into libtaskotron should decrease the amount of work and error cases that we have to deal with (assuming non-malicious users). Tim > == TODOs == > > * Change our checks to use the fedoraqa namespace > * Implement repo checking in libtaskotron > * Write docs for how to report stuff to ResultsDB > * Come up with root nodes for namespaces > _______________________________________________ > qa-devel mailing list > [email protected] > http://lists.fedoraproject.org/admin/lists/[email protected]
pgpQ3jRvCmM0w.pgp
Description: OpenPGP digital signature
_______________________________________________ qa-devel mailing list [email protected] http://lists.fedoraproject.org/admin/lists/[email protected]
