Re: Re-Scheduling Jobs for Taskotron as a User

[Josef Skladanka]

On Thu, Apr 20, 2017 at 12:07 AM, Adam Williamson <
adamw...@fedoraproject.org> wrote:

> OK, like I said, half-baked =) But wdyt?
>
>
Love it! (And I swear, it has nothing to do with the fact, that I also
thought this would be a great way to solve it in a more generic manner.)
___
qa-devel mailing list -- qa-devel@lists.fedoraproject.org
To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org

Re: Re-Scheduling Jobs for Taskotron as a User

[Adam Williamson]

On Tue, 2017-04-18 at 06:35 -0600, Tim Flink wrote:
> One of the things that seems like it is and will be a pain point for
> folks writing package-specific tasks is how to work through the times
> when there was an issue in the task and things didn't run well. At the
> moment, the only solution we have is to re-build the affected package
> or to pester an admin to re-run the trigger - neither of which is an
> ideal answer.
> 
> I was thinking about how to improve this in the near future and am
> wondering about adding a "reschedule" button to execdb jobs:
> 
>   1. authenticated user clicks on "reschedule" button
>   2. execdb makes an api call to the buildmaster to find the parameters
>  which were used for that task
>   3. using the data from 2), execdb starts a new job for just that
>  item and item type
> 
> I'm not thrilled at the idea of code duplication here between trigger
> and execdb but compared to figuring out a web interface for trigger,
> this seems like a more tractable solution for the short/medium term.
> 
> Thoughts?

Here's a Half-Baked Idea (tm):

What about a thing that's basically a simple web service that allows
FAS-authenticated people to trigger the sending of templated fedmsgs?

This could be used in various workflows like this. For this case, we'd
implement a fedmsg template which would basically say 'authenticated
user X asks that you please re-run test Y in test system Z'. Then we
could plug little bits of code into Bodhi, execdb, really anything else
you like, which would provide a button which, when clicked, calls out
to that web service to send the 'restart test' fedmsg.

The test dispatchers for Taskotron, openQA etc. would then just need to
listen out for such fedmsgs, and do whatever they felt appropriate: you
could implement policy at the dispatcher level - ignore requests from
users without certain attributes (or even stash them for manual review,
or something), accept requests from other users with different
priorities, stuff like that. For feedback...well, if the test *did* get
restarted, a fedmsg will naturally be emitted which the requesting
system could listen out for, and if the scheduler decided to reject or
sideline the request, it could emit a fedmsg saying it had done so.

OK, like I said, half-baked =) But wdyt?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
qa-devel mailing list -- qa-devel@lists.fedoraproject.org
To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org

Re: Re-Scheduling Jobs for Taskotron as a User

[Vít Ondruch]

I'd like to see something like:

$ fedpkg test

Unfortunately, internally we are using "$ rhpkg tests" to clone the test
suite into the repo 


Vít



Dne 18.4.2017 v 14:35 Tim Flink napsal(a):
> One of the things that seems like it is and will be a pain point for
> folks writing package-specific tasks is how to work through the times
> when there was an issue in the task and things didn't run well. At the
> moment, the only solution we have is to re-build the affected package
> or to pester an admin to re-run the trigger - neither of which is an
> ideal answer.
>
> I was thinking about how to improve this in the near future and am
> wondering about adding a "reschedule" button to execdb jobs:
>
>   1. authenticated user clicks on "reschedule" button
>   2. execdb makes an api call to the buildmaster to find the parameters
>  which were used for that task
>   3. using the data from 2), execdb starts a new job for just that
>  item and item type
>
> I'm not thrilled at the idea of code duplication here between trigger
> and execdb but compared to figuring out a web interface for trigger,
> this seems like a more tractable solution for the short/medium term.
>
> Thoughts?
>
> Tim
>
>
> ___
> qa-devel mailing list -- qa-devel@lists.fedoraproject.org
> To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org



signature.asc
Description: OpenPGP digital signature
___
qa-devel mailing list -- qa-devel@lists.fedoraproject.org
To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org

Re: Re-Scheduling Jobs for Taskotron as a User

[Adam Williamson]

On Tue, 2017-04-18 at 10:18 -0600, Kevin Fenzi wrote:
> On 04/18/2017 10:08 AM, Adam Williamson wrote:
> 
> > One thought: we actually have the same problem for openQA, now it's
> > running tests on updates. This isn't very visible at present so devs
> > haven't been asking about it, but I suspect as soon as the results show
> > up in Bodhi, someone will be wanting a 're-run test' button. I'm not
> > sure how far we could share any part of implementing it, though...
> 
> So what is currently triggering the tests now?
> fedmsgs when builds are tagged into specific koji tags?
> 
> I know it's not a fancy gui, but could we just untag/retag something to
> emit another fedmsg to retest?

Dunno about Taskotron package-level tests. openQA update-level tests
trigger on 'bodhi.update.edit' and 'bodhi.update.request.testing'
messages; you can trigger a re-run of *all* the tests by editing the
update, but that's a bit silly.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
qa-devel mailing list -- qa-devel@lists.fedoraproject.org
To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org

Re: Re-Scheduling Jobs for Taskotron as a User

[Kevin Fenzi]

On 04/18/2017 10:08 AM, Adam Williamson wrote:

> One thought: we actually have the same problem for openQA, now it's
> running tests on updates. This isn't very visible at present so devs
> haven't been asking about it, but I suspect as soon as the results show
> up in Bodhi, someone will be wanting a 're-run test' button. I'm not
> sure how far we could share any part of implementing it, though...

So what is currently triggering the tests now?
fedmsgs when builds are tagged into specific koji tags?

I know it's not a fancy gui, but could we just untag/retag something to
emit another fedmsg to retest?

kevin





signature.asc
Description: OpenPGP digital signature
___
qa-devel mailing list -- qa-devel@lists.fedoraproject.org
To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org

Re: Re-Scheduling Jobs for Taskotron as a User

[Adam Williamson]

On Tue, 2017-04-18 at 06:35 -0600, Tim Flink wrote:
> One of the things that seems like it is and will be a pain point for
> folks writing package-specific tasks is how to work through the times
> when there was an issue in the task and things didn't run well. At the
> moment, the only solution we have is to re-build the affected package
> or to pester an admin to re-run the trigger - neither of which is an
> ideal answer.
> 
> I was thinking about how to improve this in the near future and am
> wondering about adding a "reschedule" button to execdb jobs:
> 
>   1. authenticated user clicks on "reschedule" button
>   2. execdb makes an api call to the buildmaster to find the parameters
>  which were used for that task
>   3. using the data from 2), execdb starts a new job for just that
>  item and item type
> 
> I'm not thrilled at the idea of code duplication here between trigger
> and execdb but compared to figuring out a web interface for trigger,
> this seems like a more tractable solution for the short/medium term.
> 
> Thoughts?

One thought: we actually have the same problem for openQA, now it's
running tests on updates. This isn't very visible at present so devs
haven't been asking about it, but I suspect as soon as the results show
up in Bodhi, someone will be wanting a 're-run test' button. I'm not
sure how far we could share any part of implementing it, though...
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
qa-devel mailing list -- qa-devel@lists.fedoraproject.org
To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org

Re: Re-Scheduling Jobs for Taskotron as a User

[Tim Flink]

On Tue, 18 Apr 2017 14:53:07 +0200
Kamil Paral  wrote:

> On Tue, Apr 18, 2017 at 2:35 PM, Tim Flink  wrote:
> 
> > One of the things that seems like it is and will be a pain point for
> > folks writing package-specific tasks is how to work through the
> > times when there was an issue in the task and things didn't run
> > well. At the moment, the only solution we have is to re-build the
> > affected package or to pester an admin to re-run the trigger -
> > neither of which is an ideal answer.
> >
> > I was thinking about how to improve this in the near future and am
> > wondering about adding a "reschedule" button to execdb jobs:
> >  
> 
> Execdb web is not searchable, people will never find their job. It
> makes a lot of sense to have the button in execdb, because that's
> where we want to show people crashed jobs (those will not appear
> neither in resultsdb nor in Bodhi). So thumbs up to that idea. But
> the search seems to be a prerequisite to this. Or we need to make
> sure crashed jobs are sent out with direct links to execdb to the
> relevant maintainers, if that turns out to be easier.

Yeah, it's not an ideal solution but in my mind, it's better than
asking folks to rebuild just to get the task re-run.

> >   1. authenticated user clicks on "reschedule" button
> >  
> 
> Authentication itself sounds like a non-trivial issue. I wonder if we
> can avoid it at least temporarily. Maybe allow anyone to click the
> button, but limit the number of times it can be pressed (e.g. max 5
> times)? Also introduce a timeout after the press (max once per hour
> since the last re-run), and don't allow to re-run old jobs (e.g.
> older than 24 hours)?
> 
> Maybe authentication will be easier after all :) Maybe we don't even
> need to check package permissions in the beginning, just making sure
> the person has a valid FAS (cla+1) account.

I was hoping that we'd be able to auth against ipsilon without much
added effort. Ideally we'd restrict things to someone with ACLs to the
involved package but that's far from required.

cla+1 should be good enough for now, I think. Limiting it to
packagers could also be reasonable but I'm not sure it's going to be a
big deal so long as we can keep track of who's doing the reschedules
through at least a log file.

Tim

> 
> >   2. execdb makes an api call to the buildmaster to find the
> > parameters which were used for that task
> >   3. using the data from 2), execdb starts a new job for just that
> >  item and item type
> >
> > I'm not thrilled at the idea of code duplication here between
> > trigger and execdb but compared to figuring out a web interface for
> > trigger, this seems like a more tractable solution for the
> > short/medium term.
> >
> > Thoughts?
> >
> > Tim
> >
> > ___
> > qa-devel mailing list -- qa-devel@lists.fedoraproject.org
> > To unsubscribe send an email to
> > qa-devel-le...@lists.fedoraproject.org
> >
> >  



pgpQbLfl5KjHY.pgp
Description: OpenPGP digital signature
___
qa-devel mailing list -- qa-devel@lists.fedoraproject.org
To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org

Re: Re-Scheduling Jobs for Taskotron as a User

[Kamil Paral]

On Tue, Apr 18, 2017 at 2:35 PM, Tim Flink  wrote:

> One of the things that seems like it is and will be a pain point for
> folks writing package-specific tasks is how to work through the times
> when there was an issue in the task and things didn't run well. At the
> moment, the only solution we have is to re-build the affected package
> or to pester an admin to re-run the trigger - neither of which is an
> ideal answer.
>
> I was thinking about how to improve this in the near future and am
> wondering about adding a "reschedule" button to execdb jobs:
>

Execdb web is not searchable, people will never find their job. It makes a
lot of sense to have the button in execdb, because that's where we want to
show people crashed jobs (those will not appear neither in resultsdb nor in
Bodhi). So thumbs up to that idea. But the search seems to be a
prerequisite to this. Or we need to make sure crashed jobs are sent out
with direct links to execdb to the relevant maintainers, if that turns out
to be easier.


>   1. authenticated user clicks on "reschedule" button
>

Authentication itself sounds like a non-trivial issue. I wonder if we can
avoid it at least temporarily. Maybe allow anyone to click the button, but
limit the number of times it can be pressed (e.g. max 5 times)? Also
introduce a timeout after the press (max once per hour since the last
re-run), and don't allow to re-run old jobs (e.g. older than 24 hours)?

Maybe authentication will be easier after all :) Maybe we don't even need
to check package permissions in the beginning, just making sure the person
has a valid FAS (cla+1) account.



>   2. execdb makes an api call to the buildmaster to find the parameters
>  which were used for that task
>   3. using the data from 2), execdb starts a new job for just that
>  item and item type
>
> I'm not thrilled at the idea of code duplication here between trigger
> and execdb but compared to figuring out a web interface for trigger,
> this seems like a more tractable solution for the short/medium term.
>
> Thoughts?
>
> Tim
>
> ___
> qa-devel mailing list -- qa-devel@lists.fedoraproject.org
> To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org
>
>
___
qa-devel mailing list -- qa-devel@lists.fedoraproject.org
To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org