> This is an initial take on stuff that was discussed in person during Tim's
> stay in Brno. Sending to the list for additional discussion/fine-tuning.
>
> = What =
>
> Talking rpmgrill-like checks, there will be a need to be able to facilitate
> some kind of structure for representing that a check is composed of multiple
> subchecks, for example:
>
> check - FAILED
> subcheck1 - PASSED
> subcheck2 - PASSED
> subcheck3 - FAILED
> subcheck4 - PASSED
>
> !IMPORTANT: ResultsDB will not be responsible for computing the result value
> for an "upper level" Result from the subchecks - this is the check's (check
> developer's) responsibility.
I just realized this can go hand-in-hand with the new create_report directive
that we introduced lately. If the generated report contains all the reported
results and links to respective log files (if there are separate log files for
individual subchecks, might or might not be true), this way even if the user is
notified about the subcheck3 result, he can still click through and see the
other check/subcheck results and their logs easily as well.
>
> This could (should?) be done on two levels:
> * physicall nesting the Results as such in the database structure
> * namespacing Testcases
>
> For the start, we decided to go with the simplistic approach of nesting the
> Testcases via a simple namespacing - thus allowing a frontend/query tool to
> reconstruct the structure at least to some extent e.g. by relying on a
> premise,
> that Results that are a part of one Job can be converted to a tree-like
> structure, based on the Testcase namespacing, at least to some extent, if
> needed.
>
>
> == Namespace structure ==
>
> We'll be providing some top-level namespaces (list not yet final):
> * app
This seems obvious at first, but the more I think about it, the more I'm not
sure whether it wouldn't be clearer to use something like qa.testdays instead
of app.testdays and team.cloud.tunir instead of app.tunir. Because at least
it's obvious who stands behind it and we don't need to decide which "app" is
high-profile enough to be put into "app" top namespace instead of the generic
user/team namespaces. But if we have a clear idea what goes here and what
doesn't, I'm OK with it.
> * fedoraqa
I'd probably use just "qa". "fedora" seems superfluous. Or even "team.qa", see
below.
> * package
Maybe just "pkg"?
> * scratch (?)
I'd also add "fas"/"user" and "team". This way everyone can use his/her own
namespace (like user home on wiki) for experimenting or custom tasks, and we
also make team ownership clearer ("team.infra.task" instead of
"user.pingou.task"). With this, we might not even need "scratch", because
everyone can play is his/her own user namespace (but we might find some other
uses).
>
> These will the further split to facilitate for a finer level of granularity,
> e.g.:
>
> app
> testdays
> powermanagement
> pm-suspendr
> fedoraqa
> depcheck
> rpmgrill
> package
> <pkgname>
> unit
> func
>
> Everything below the top-level will be 100% user defined. We might have
> recommendations for specific namespaces (like package.<pkgname>), but we
> won't
> be enforcing them.
By top-level you mean whole "pkg.<pkgname>" in this case, right? Because it
would be weird to allow people to arbitrarily create "pkg.foo" namespace if
there's no "foo" package in Fedora (and would introduce issues in the future,
if such package appeared). So "pkg.<pkgname>" would be the fixed part, and
anything below would be the flexible part, right? The same with
"user.<username>" and "team.<teamname>".
>
> The structure will be implemented (at least in the initial implementation)
> just
> via the Testcase.name attribute in the DB, using dots as a separator. Later
> on,
> we can easily add an easy way of using wildcards for searching (e.g.
> app.testdays.*.pm-suspendr)
>
> !IMPORTANT: the namespaces are not to be used to represent "additional data"
> about the underlying result such as architecture, item under test, etc.
> This is what the Result's extra-data (ResultData) is there for.
>
> NOTE: Although we do not encourage to store the results to the finest
> granularity "just because" (e.g. individual results of a unittest testsuite),
> we leave it to the check-developer's judgement. If there is a usecase for it,
> let them do it, we don't care, as long as the DB is not extremely overloaded.
This is a very good point and in the documentation I'd like to stress out
"please don't create dozens of namespaces needlessly". We should explain when
it makes sense to use subcheck namespaces and when it does not. For example, if
"rpmgrill" executes many different checks and we consider some of them
release-critical and some of them not, which means we want to listen for them
individually over fedpkg (and package maintainers as well), it makes sense to
have qa.rpmgrill.rpm_sanity, qa.rpmgrill.rpmlint, qa.rpmgrill.security, etc.
However, if openssh test suite has 27 steps/test functions, it does not make
sense to create pkg.openssh.step1 through pkg.openssh.step27 namespaces, if we
only care about the final result, or if we process it in a single place only.
If developers want the step separation but they process it in a single place,
it's much better to create a parseable ResultYAML/JUnit file with a
well-defined name as one of the artifacts (we can do that automatically for
every task), and they can download it from the output directory and parse it.
It's the same amount of work and it does not put 27 times heavier demand on our
database and fedmsg.
>
>
> == Authentication/Authorization ==
>
> We'll be continuing with the "expect no malice" approach we have right now.
> There will be just a simple limitation in libtaskotron:
>
> check git clone
> if cloned: only allow non-pkg namespace if __our__ repo
> else: do whatever, don't care
>
> in libtaskotron:
> check the git checkout like listed above
> have whitelisted napespace repos in config
>
> !FIXME: the mechanism above is just copied from tflink's notes, I can't
> remember the details :/
I'm not sure I understand this correctly, but I believe the idea was to give
out namespace write-access based on git source URL. So tasks from dist-git
system would only have write access to pkg.<pkgname>, where <pkgname> matches
the package name in their dist-git URL. Tasks from QA repos would have
write-access to qa namespace. Etc.
_______________________________________________
qa-devel mailing list
[email protected]
http://lists.fedoraproject.org/admin/lists/[email protected]
