Re: New automated test coverage: openQA tests of critical path updates

2017-03-06 Thread Adam Williamson
On Mon, 2017-02-27 at 10:22 -0800, Adam Williamson wrote: > Hi folks! > > I am currently rolling out some changes to the Fedora openQA deployment > which enable a new testing workflow. From now on, a subset of openQA > tests should be run automatically on every critpath update, both on > initial

Re: New automated test coverage: openQA tests of critical path updates

2017-03-03 Thread Kamil Paral
> On Thu, 2017-03-02 at 04:31 -0500, Kamil Paral wrote: > > > The job can - and already does - log the exact packages it actually > > > got, but I don't think there's an easy way for it to take the > > > 'last_modified' date for the update at the time it does the download. > > > > I don't know

Re: New automated test coverage: openQA tests of critical path updates

2017-03-02 Thread Adam Williamson
On Thu, 2017-03-02 at 04:31 -0500, Kamil Paral wrote: > > The job can - and already does - log the exact packages it actually > > got, but I don't think there's an easy way for it to take the > > 'last_modified' date for the update at the time it does the download. > > I don't know how you

Re: New automated test coverage: openQA tests of critical path updates

2017-03-02 Thread Kamil Paral
> > There's one important thing we need to do first, though. Bodhi ID > > doesn't identify the thing tested uniquely, because Bodhi updates are > > mutable (and the ID is kept). So Bodhi (or any gating tools) can't > > rely on just retrieving the latest result for a particular Bodhi ID > > and

Re: New automated test coverage: openQA tests of critical path updates

2017-03-02 Thread Jan Sedlak
2017-03-01 18:04 GMT+01:00 Adam Williamson : > I'm not so sure it's really necessary, and doing it is actually tricky > for openQA. Only the openQA job itself knows what packages it actually > tested, and it doesn't have an easy way to get the associated > timestamp.

Re: New automated test coverage: openQA tests of critical path updates

2017-03-01 Thread Adam Williamson
On Wed, 2017-03-01 at 11:18 -0500, Kamil Paral wrote: > So my first thought was to recommend you to also publish just > type=koji_build results and finish this transition. But then I > realized that's wrong. OpenQA operates completely different than the > aforementioned tasks do. We operate on

Re: New automated test coverage: openQA tests of critical path updates

2017-03-01 Thread Kamil Paral
> Hi folks! > > I am currently rolling out some changes to the Fedora openQA deployment > which enable a new testing workflow. From now on, a subset of openQA > tests should be run automatically on every critpath update, both on > initial submission and on any edit of the update. > > For the