Re: The future of loopabull
On Thu, May 28, 2020 at 02:18:38PM +0200, Pierre-Yves Chibon wrote: > Good Morning Everyone, > > I know this question has already been raised a few times, but I think we > should > raise it once more: what do we see as future for loopabull? > > It is currently triggered on 4 topics (3 from prod and 1 from stg) to do > basically > three actions: > - Flag commit successfully built in koji, in other words it adds these flags > to dist-git: > > https://src.fedoraproject.org/rpms/mingw-filesystem/c/717f2a929bd25b62a0427e8e5c3792a0939dbfce > - Flag when the Fedora CI start testing a PR > - Flag when the Fedora CI finished testing a PR (and thus reports Pass/Fail) > > Upstream released yesterday a 0.0.7 release which brings supports for > fedora-messaging (contributed by your servitor). > Looking at the code, it should be python3 compatible, but it doesn't say > specifically in the setup.py and I honestly don't remember if I've tested that > or not. > The package has been orphaned in Fedora for over 10 months and has thus been > retired. > > I've had a chat with upstream yesterday and they are still interested in the > project but more as a pet project and their time is just like the rest of us, > limited for pet projects these days. > That being said the code base is really quite small and involves technologies > we're already using in other places (python-pika, celery, rabbitmq, > ansible...) > so there isn't really anything new there. > > One of its limitation currently is with secrets, how to pass/specify them. > This is something we could circumvent via ansible-vault or so, but it needs a > little investigation. > > I basically see three ways forward with this: > - We continue with loopabull and we need to check its python3 support, how to > deal with secrets, if we can get it to run in openshift & so on. > - We look for something else, similar. The requirements being: > - Run a task when seeing a message in our message bus > - Handle secrets > - Scalable up/down > - Runnable in openshift is a bonus > - Preferably in a language we can debug (python++, potentially rust) > - We write something that fits our needs and requirements > > > Would you know something that fits our requirements and that we could just > run? > If not, do you have a preferred way between options #1 and #3? So I took a few hours and started poking at this. I've ended up with: https://pagure.io/fedora-infra/toddlers (a bunch of small programs running around: toddlers :)) It's basically, one fedora-messaging consumer and plugins. Some of its advantages: - We can run it in openshift easily and we should be able to scale it up/down as needed - Managing secrets is just like managing secrets for all our apps in openshift. - Relies on our main rabbitmq server and thus has monitoring (while loopabull ran its own, which granted we could monitor but that's something we've not done while we have this in place for our main cluster and easily configurable via ansible). I would still like to add some unit-tests for this but from empirical testing (ie: running it locally) it is doing what it should be doing and the two/three tasks loopabull have already been ported. Let me know what you think of it! :) Pierre ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: The future of loopabull
On Mon, 1 Jun 2020 at 15:37, Pierre-Yves Chibon wrote: > On Fri, May 29, 2020 at 03:18:58PM +0200, Clement Verna wrote: > >On Thu, 28 May 2020 at 14:27, Pierre-Yves Chibon <[1] > pin...@pingoured.fr> > >wrote: > > > > Good Morning Everyone, > > > > I know this question has already been raised a few times, but I > think we > > should > > raise it once more: what do we see as future for loopabull? > > > > It is currently triggered on 4 topics (3 from prod and 1 from stg) > to do > > basically > > three actions: > > - Flag commit successfully built in koji, in other words it adds > these > > flags > >to dist-git: > > > > [2] > https://src.fedoraproject.org/rpms/mingw-filesystem/c/717f2a929bd25b62a0427e8e5c3792a0939dbfce > > - Flag when the Fedora CI start testing a PR > > - Flag when the Fedora CI finished testing a PR (and thus reports > > Pass/Fail) > > > > Upstream released yesterday a 0.0.7 release which brings supports > for > > fedora-messaging (contributed by your servitor). > > Looking at the code, it should be python3 compatible, but it > doesn't say > > specifically in the setup.py and I honestly don't remember if I've > > tested that > > or not. > > The package has been orphaned in Fedora for over 10 months and has > thus > > been > > retired. > > > > I've had a chat with upstream yesterday and they are still > interested in > > the > > project but more as a pet project and their time is just like the > rest > > of us, > > limited for pet projects these days. > > That being said the code base is really quite small and involves > > technologies > > we're already using in other places (python-pika, celery, rabbitmq, > > ansible...) > > so there isn't really anything new there. > > > > One of its limitation currently is with secrets, how to pass/specify > > them. > > This is something we could circumvent via ansible-vault or so, but > it > > needs a > > little investigation. > > > > I basically see three ways forward with this: > > - We continue with loopabull and we need to check its python3 > support, > > how to > >deal with secrets, if we can get it to run in openshift & so on. > > - We look for something else, similar. The requirements being: > >- Run a task when seeing a message in our message bus > >- Handle secrets > >- Scalable up/down > >- Runnable in openshift is a bonus > >- Preferably in a language we can debug (python++, potentially > rust) > > - We write something that fits our needs and requirements > > > >There is a PR[0] in fedora-messaging to add a 'run' callback that > would > >let you execute any command, I think that might be a nice solution > and I > >think it would meet most of the requirements. > > > >[0] - [3]https://github.com/fedora-infra/fedora-messaging/pull/163 > > Isn't that the equivalent of having us write a custom fedora-messaging > consumer > for each task we want to automate? > Yes but without all the boilerplate needed for each consumer. > In a way I like this, it's simple(r), really straight forward, constraint > and > self-contained. There is no risk that a previous task prevents a following > one > to be executed. > On the other hand it also means that if we move to, say fed-messaging, in > the > future, we will have to convert more consumers. > > If that's a trade off we're willing to live with, then I'm ok with it as > well. > I don't have a strong opinion here :-) > > > Pierre > ___ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to > infrastructure-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org > ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: The future of loopabull
On Fri, May 29, 2020 at 03:18:58PM +0200, Clement Verna wrote: >On Thu, 28 May 2020 at 14:27, Pierre-Yves Chibon <[1]pin...@pingoured.fr> >wrote: > > Good Morning Everyone, > > I know this question has already been raised a few times, but I think we > should > raise it once more: what do we see as future for loopabull? > > It is currently triggered on 4 topics (3 from prod and 1 from stg) to do > basically > three actions: > - Flag commit successfully built in koji, in other words it adds these > flags > to dist-git: > > > [2]https://src.fedoraproject.org/rpms/mingw-filesystem/c/717f2a929bd25b62a0427e8e5c3792a0939dbfce > - Flag when the Fedora CI start testing a PR > - Flag when the Fedora CI finished testing a PR (and thus reports > Pass/Fail) > > Upstream released yesterday a 0.0.7 release which brings supports for > fedora-messaging (contributed by your servitor). > Looking at the code, it should be python3 compatible, but it doesn't say > specifically in the setup.py and I honestly don't remember if I've > tested that > or not. > The package has been orphaned in Fedora for over 10 months and has thus > been > retired. > > I've had a chat with upstream yesterday and they are still interested in > the > project but more as a pet project and their time is just like the rest > of us, > limited for pet projects these days. > That being said the code base is really quite small and involves > technologies > we're already using in other places (python-pika, celery, rabbitmq, > ansible...) > so there isn't really anything new there. > > One of its limitation currently is with secrets, how to pass/specify > them. > This is something we could circumvent via ansible-vault or so, but it > needs a > little investigation. > > I basically see three ways forward with this: > - We continue with loopabull and we need to check its python3 support, > how to > deal with secrets, if we can get it to run in openshift & so on. > - We look for something else, similar. The requirements being: > - Run a task when seeing a message in our message bus > - Handle secrets > - Scalable up/down > - Runnable in openshift is a bonus > - Preferably in a language we can debug (python++, potentially rust) > - We write something that fits our needs and requirements > >There is a PR[0] in fedora-messaging to add a 'run' callback that would >let you execute any command, I think that might be a nice solution and I >think it would meet most of the requirements. > >[0] - [3]https://github.com/fedora-infra/fedora-messaging/pull/163 Isn't that the equivalent of having us write a custom fedora-messaging consumer for each task we want to automate? In a way I like this, it's simple(r), really straight forward, constraint and self-contained. There is no risk that a previous task prevents a following one to be executed. On the other hand it also means that if we move to, say fed-messaging, in the future, we will have to convert more consumers. If that's a trade off we're willing to live with, then I'm ok with it as well. Pierre ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: The future of loopabull
On Thu, 28 May 2020 at 14:27, Pierre-Yves Chibon wrote: > Good Morning Everyone, > > I know this question has already been raised a few times, but I think we > should > raise it once more: what do we see as future for loopabull? > > It is currently triggered on 4 topics (3 from prod and 1 from stg) to do > basically > three actions: > - Flag commit successfully built in koji, in other words it adds these > flags > to dist-git: > > https://src.fedoraproject.org/rpms/mingw-filesystem/c/717f2a929bd25b62a0427e8e5c3792a0939dbfce > - Flag when the Fedora CI start testing a PR > - Flag when the Fedora CI finished testing a PR (and thus reports > Pass/Fail) > > Upstream released yesterday a 0.0.7 release which brings supports for > fedora-messaging (contributed by your servitor). > Looking at the code, it should be python3 compatible, but it doesn't say > specifically in the setup.py and I honestly don't remember if I've tested > that > or not. > The package has been orphaned in Fedora for over 10 months and has thus > been > retired. > > I've had a chat with upstream yesterday and they are still interested in > the > project but more as a pet project and their time is just like the rest of > us, > limited for pet projects these days. > That being said the code base is really quite small and involves > technologies > we're already using in other places (python-pika, celery, rabbitmq, > ansible...) > so there isn't really anything new there. > > One of its limitation currently is with secrets, how to pass/specify them. > This is something we could circumvent via ansible-vault or so, but it > needs a > little investigation. > > I basically see three ways forward with this: > - We continue with loopabull and we need to check its python3 support, how > to > deal with secrets, if we can get it to run in openshift & so on. > - We look for something else, similar. The requirements being: > - Run a task when seeing a message in our message bus > - Handle secrets > - Scalable up/down > - Runnable in openshift is a bonus > - Preferably in a language we can debug (python++, potentially rust) > - We write something that fits our needs and requirements > There is a PR[0] in fedora-messaging to add a 'run' callback that would let you execute any command, I think that might be a nice solution and I think it would meet most of the requirements. [0] - https://github.com/fedora-infra/fedora-messaging/pull/163 > > > Would you know something that fits our requirements and that we could just > run? > If not, do you have a preferred way between options #1 and #3? > > > Thanks for your thoughts, > > Pierre > ___ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to > infrastructure-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org > ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: The future of loopabull
On Thu, May 28, 2020 at 10:31 AM Adam Williamson wrote: > > Why, though? > > I quite like the design of small consumer components to do specific > jobs in response to messages. Lots of the bits I maintain work that > way. Why is it necessarily better for Koji to grow the ability to > change stuff in dist-git than to have a little go-between to do the > job? One nice property of doing these things as separate services is that they are more resilient to cascading failures. Tomas found some problems recently with Koji failing to retry sending messages when a messagebus connection gets into a certain (rare) state where it is unavailable but proton is confused. Koji ends up dropping those messages on the floor, but in the meantime there is more pressure on the koji-hub application. I could envision a worse scenario where Koji has to send even more "events" reliably to a bunch of other services that could be responding very slowly, etc. The same thing happens with Pagure Git hooks - unfortunately the more we pile in there, the slower everything becomes. - Ken ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: The future of loopabull
On Thu, May 28, 2020 at 12:31 PM Adam Williamson wrote: > > On Thu, 2020-05-28 at 08:30 -0400, Neal Gompa wrote: > > On Thu, May 28, 2020 at 8:19 AM Pierre-Yves Chibon > > wrote: > > > Good Morning Everyone, > > > > > > I know this question has already been raised a few times, but I think we > > > should > > > raise it once more: what do we see as future for loopabull? > > > > > > It is currently triggered on 4 topics (3 from prod and 1 from stg) to do > > > basically > > > three actions: > > > - Flag commit successfully built in koji, in other words it adds these > > > flags > > > to dist-git: > > > > > > https://src.fedoraproject.org/rpms/mingw-filesystem/c/717f2a929bd25b62a0427e8e5c3792a0939dbfce > > > - Flag when the Fedora CI start testing a PR > > > - Flag when the Fedora CI finished testing a PR (and thus reports > > > Pass/Fail) > > > > > > > Wait, wait, wait, what?! These aren't things natively supported by the > > software we use (Koji and Zuul)? It seems like the place to start here > > would be to move this functionality to the right places. A koji plugin > > and an extension for the Zuul pagure driver to do those things would > > make a ton more sense here... > > Why, though? > > I quite like the design of small consumer components to do specific > jobs in response to messages. Lots of the bits I maintain work that > way. Why is it necessarily better for Koji to grow the ability to > change stuff in dist-git than to have a little go-between to do the > job? If it's discoverable, I don't care. But I had no idea that Koji and Zuul weren't responsible for those things. As for loopabull, I don't particularly see a problem with keeping it. There's nothing wrong with hobby-maintained software. After all, we're a hobby-maintained distribution. :) -- 真実はいつも一つ！/ Always, there's only one truth! ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: The future of loopabull
On Thu, 2020-05-28 at 08:30 -0400, Neal Gompa wrote: > On Thu, May 28, 2020 at 8:19 AM Pierre-Yves Chibon > wrote: > > Good Morning Everyone, > > > > I know this question has already been raised a few times, but I think we > > should > > raise it once more: what do we see as future for loopabull? > > > > It is currently triggered on 4 topics (3 from prod and 1 from stg) to do > > basically > > three actions: > > - Flag commit successfully built in koji, in other words it adds these flags > > to dist-git: > > > > https://src.fedoraproject.org/rpms/mingw-filesystem/c/717f2a929bd25b62a0427e8e5c3792a0939dbfce > > - Flag when the Fedora CI start testing a PR > > - Flag when the Fedora CI finished testing a PR (and thus reports Pass/Fail) > > > > Wait, wait, wait, what?! These aren't things natively supported by the > software we use (Koji and Zuul)? It seems like the place to start here > would be to move this functionality to the right places. A koji plugin > and an extension for the Zuul pagure driver to do those things would > make a ton more sense here... Why, though? I quite like the design of small consumer components to do specific jobs in response to messages. Lots of the bits I maintain work that way. Why is it necessarily better for Koji to grow the ability to change stuff in dist-git than to have a little go-between to do the job? -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: The future of loopabull
On Thu, May 28, 2020 at 08:30:02AM -0400, Neal Gompa wrote: > On Thu, May 28, 2020 at 8:19 AM Pierre-Yves Chibon > wrote: > > > > Good Morning Everyone, > > > > I know this question has already been raised a few times, but I think we > > should > > raise it once more: what do we see as future for loopabull? > > > > It is currently triggered on 4 topics (3 from prod and 1 from stg) to do > > basically > > three actions: > > - Flag commit successfully built in koji, in other words it adds these flags > > to dist-git: > > > > https://src.fedoraproject.org/rpms/mingw-filesystem/c/717f2a929bd25b62a0427e8e5c3792a0939dbfce > > - Flag when the Fedora CI start testing a PR > > - Flag when the Fedora CI finished testing a PR (and thus reports Pass/Fail) > > > > Wait, wait, wait, what?! These aren't things natively supported by the > software we use (Koji and Zuul)? It seems like the place to start here > would be to move this functionality to the right places. A koji plugin > and an extension for the Zuul pagure driver to do those things would > make a ton more sense here... Loopabull executes any playbook on a message, the fact that we're using it only for that is a mere accident, it's meant to be much more than that. We could in fact use it to do some of the releng tasks (that was one of the original idea for it), we just haven't got around to do that. Zuul may be an option here, though wouldn't it have the same issue as loopabull wrt secrets? Pierre ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: The future of loopabull
On Thu, May 28, 2020 at 8:19 AM Pierre-Yves Chibon wrote: > > Good Morning Everyone, > > I know this question has already been raised a few times, but I think we > should > raise it once more: what do we see as future for loopabull? > > It is currently triggered on 4 topics (3 from prod and 1 from stg) to do > basically > three actions: > - Flag commit successfully built in koji, in other words it adds these flags > to dist-git: > > https://src.fedoraproject.org/rpms/mingw-filesystem/c/717f2a929bd25b62a0427e8e5c3792a0939dbfce > - Flag when the Fedora CI start testing a PR > - Flag when the Fedora CI finished testing a PR (and thus reports Pass/Fail) > Wait, wait, wait, what?! These aren't things natively supported by the software we use (Koji and Zuul)? It seems like the place to start here would be to move this functionality to the right places. A koji plugin and an extension for the Zuul pagure driver to do those things would make a ton more sense here... -- 真実はいつも一つ！/ Always, there's only one truth! ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
The future of loopabull
Good Morning Everyone, I know this question has already been raised a few times, but I think we should raise it once more: what do we see as future for loopabull? It is currently triggered on 4 topics (3 from prod and 1 from stg) to do basically three actions: - Flag commit successfully built in koji, in other words it adds these flags to dist-git: https://src.fedoraproject.org/rpms/mingw-filesystem/c/717f2a929bd25b62a0427e8e5c3792a0939dbfce - Flag when the Fedora CI start testing a PR - Flag when the Fedora CI finished testing a PR (and thus reports Pass/Fail) Upstream released yesterday a 0.0.7 release which brings supports for fedora-messaging (contributed by your servitor). Looking at the code, it should be python3 compatible, but it doesn't say specifically in the setup.py and I honestly don't remember if I've tested that or not. The package has been orphaned in Fedora for over 10 months and has thus been retired. I've had a chat with upstream yesterday and they are still interested in the project but more as a pet project and their time is just like the rest of us, limited for pet projects these days. That being said the code base is really quite small and involves technologies we're already using in other places (python-pika, celery, rabbitmq, ansible...) so there isn't really anything new there. One of its limitation currently is with secrets, how to pass/specify them. This is something we could circumvent via ansible-vault or so, but it needs a little investigation. I basically see three ways forward with this: - We continue with loopabull and we need to check its python3 support, how to deal with secrets, if we can get it to run in openshift & so on. - We look for something else, similar. The requirements being: - Run a task when seeing a message in our message bus - Handle secrets - Scalable up/down - Runnable in openshift is a bonus - Preferably in a language we can debug (python++, potentially rust) - We write something that fits our needs and requirements Would you know something that fits our requirements and that we could just run? If not, do you have a preferred way between options #1 and #3? Thanks for your thoughts, Pierre ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
