This is an initial take on stuff that was discussed in person during Tim's
stay in Brno. Sending to the list for additional discussion/fine-tuning.
= What =
Talking rpmgrill-like checks, there will be a need to be able to facilitate
some kind of structure for representing that a check is composed of multiple
subchecks, for example:
check - FAILED
subcheck1 - PASSED
subcheck2 - PASSED
subcheck3 - FAILED
subcheck4 - PASSED
!IMPORTANT: ResultsDB will not be responsible for computing the result value
for an "upper level" Result from the subchecks - this is the check's (check
developer's) responsibility.
This could (should?) be done on two levels:
* physicall nesting the Results as such in the database structure
* namespacing Testcases
For the start, we decided to go with the simplistic approach of nesting the
Testcases via a simple namespacing - thus allowing a frontend/query tool to
reconstruct the structure at least to some extent e.g. by relying on a premise,
that Results that are a part of one Job can be converted to a tree-like
structure, based on the Testcase namespacing, at least to some extent, if
needed.
== Namespace structure ==
We'll be providing some top-level namespaces (list not yet final):
* app
* fedoraqa
* package
* scratch (?)
These will the further split to facilitate for a finer level of granularity,
e.g.:
app
testdays
powermanagement
pm-suspendr
fedoraqa
depcheck
rpmgrill
package
<pkgname>
unit
func
Everything below the top-level will be 100% user defined. We might have
recommendations for specific namespaces (like package.<pkgname>), but we won't
be enforcing them.
The structure will be implemented (at least in the initial implementation) just
via the Testcase.name attribute in the DB, using dots as a separator. Later on,
we can easily add an easy way of using wildcards for searching (e.g.
app.testdays.*.pm-suspendr)
!IMPORTANT: the namespaces are not to be used to represent "additional data"
about the underlying result such as architecture, item under test, etc.
This is what the Result's extra-data (ResultData) is there for.
NOTE: Although we do not encourage to store the results to the finest
granularity "just because" (e.g. individual results of a unittest testsuite),
we leave it to the check-developer's judgement. If there is a usecase for it,
let them do it, we don't care, as long as the DB is not extremely overloaded.
== Authentication/Authorization ==
We'll be continuing with the "expect no malice" approach we have right now.
There will be just a simple limitation in libtaskotron:
check git clone
if cloned: only allow non-pkg namespace if __our__ repo
else: do whatever, don't care
in libtaskotron:
check the git checkout like listed above
have whitelisted napespace repos in config
!FIXME: the mechanism above is just copied from tflink's notes, I can't
remember the details :/
== TODOs ==
* Change our checks to use the fedoraqa namespace
* Implement repo checking in libtaskotron
* Write docs for how to report stuff to ResultsDB
* Come up with root nodes for namespaces
_______________________________________________
qa-devel mailing list
qa-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/qa-devel@lists.fedoraproject.org
