On Mon, Nov 28, 2016 at 6:48 PM, Adam Williamson <adamw...@fedoraproject.org
> wrote:

> On Mon, 2016-11-28 at 09:40 -0800, Adam Williamson wrote:
> > The validator/submitter component would be responsible for watching out
> > for new composes and keeping track of tests and 'test environments' (if
> > we keep that concept); it would have an API with endpoints you could
> > query for this kind of information in order to construct a result
> > submission, and for submitting results in some kind of defined form. On
> > receiving a result it would validate it according to some schemas that
> > admins of the system could configure (to ensure the report is for a
> > known compose, image, test and test environment, and do some checking
> > of stuff like the result status, user who submitted the result, comment
> > content, stuff like that). Then it'd forward the result to resultsdb.
> It occurs to me that it's possible resultsdb might be designed to do
> all this already, or it might make sense to amend resultsdb to do all
> or some of it; if that's the case, resultsdb folks, please do jump in
> and suggest it :)

That's what I thought, when reading the proposal - the "Submitter" seems
like an unnecessary layer, to some extent - submitting stuff to resultsdb
is pretty easy.
What resultsdb is not doing now, though is the data validation - let's say
you wanted to check that specific fields are set (on top of what resultsdb
requires, which basically is just testcase and outcome) - that can be done
in resultsdb (there is a diff with that functionality), but at the moment
only on global level. So it might not necessarily make sense to set e.g.
'compose' as a required field for the whole resultsdb, since
testday-related results might not even have that.
So if this is what you wanted to do (data validation), it might be a good
idea to have that submitter middleware. Or (and I'm not sure it's the
better solution) I could try and make that configuration more granular, so
you could set the requirements e.g. per namespace, thus effectively
allowing setting the constraints even per testcase. But that would need
even more though - should the constraints be inherited from the upper
layers? How about when all but one testcases in a namespace need to have
parameter X, but for the one, it does not make sense? (Probably a design
error, but needs to be thought-through in the design phase).

So, even though resultsDB could do that, it is borderline "too smart" for
it (I really want to keep any semantics out of ResultsDB). I'm not
necessarily against it (especially if we end up wanting that on more
places), but until now, we more or less worked with "clients that submits
data makes sure all required fields are set" i.e "it's not resultsdb's
place to say what is or is not required for a specific usecase". I'm not
against the change, but at least for the first implementation (of the
Release validation NG) I'd vote for the middleware solution. We can add the
data validation functionality to ResultsDB later on, when we have a more
concrete idea.

Makes sense?

qa-devel mailing list -- qa-devel@lists.fedoraproject.org
To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org

Reply via email to