Re: [gentoo-dev] Request for help: distributed pull request CI (pkgcheck)

[nado]

December 7, 2016 7:57 PM, "Michał Górny"  wrote:

> Can you think of any tools that could help me get the task done easily
> and with as little of reinventing the wheel as possible? Right now, I
> have just a lot of trivial shell script that checks pull request for
> changes, checks them out, runs 'pmaint regen', pkgcheck, publishes all
> kinds of results and statuses, and also compares QA results to
> determine new failures created by the PR [1].

If this is already shell based, why not going for something like an ansible 
automated task or alike ? This way, helper would almost only need to give an 
ssh key to setup everything. Of course for managing the load it would need some 
more tinkering, and I’m not sure it would be totally flexible and expandable.

--
Corentin “Nado” Pazdera

Re: [gentoo-dev] Request for help: distributed pull request CI (pkgcheck)

[Mathy Vanvoorden]

Hi,

I have some experience with CI and am willing to help out, I'll talk to you
on IRC.

Mathy

On Dec 7, 2016 19:57, "Michał Górny"  wrote:

> Hello, everyone.
>
> TL;DR: I'm looking for some help to get pull-request CI running
> distributed, i.e. on multiple hosts.
>
>
> Quick history
> -
>
> The repo-mirror-ci project pretty much started as a simple cronjob
> running on hardware contributed to the purpose by Todd Goodman
> (considering how much this hardware has done for Gentoo, I'd suggest
> giving him a honorary citizenship of Gentoo or something like
> that ;-)). I can't say they were terrible but they weren't perfect
> either. Due to lack of time, I mostly focused on improving them
> moderately and adding new features.
>
> Today, everything is still running on the same hardware. While
> the hardware is real good, and I'm doing some effort to improve
> the performance of the tools, I think it's quite close to its limits.
>
> Right now, it is running three tasks every 20 minutes: updating repo
> mirrors, running pkgcheck on Gentoo and processing one pull request.
> However, if those tasks delay beyond those 20 minutes, the poor man's
> scheduling based on cron + lockfiles may cause some of the executions
> to be skipped.
>
> In other words, the best case is processing 3 pull requests / h. This
> is starting to be no longer sufficient, and causing significant delays
> during periods of high contributor activity -- while my goal would be to
> provide the results ASAP -- 10-20 minutes would be perfect.
>
>
> The goal and needs
> --
>
> I think the best way forward would be to start distributing tasks
> between more systems. While I don't really have time to make whole
> repo-mirror-ci distributed, I think it should be feasible to start
> splitting pull request processing out of it.
>
> I would use some help to achieve that, esp. wrt distributed task
> scheduling.
>
> Can you think of any tools that could help me get the task done easily
> and with as little of reinventing the wheel as possible? Right now, I
> have just a lot of trivial shell script that checks pull request for
> changes, checks them out, runs 'pmaint regen', pkgcheck, publishes all
> kinds of results and statuses, and also compares QA results to
> determine new failures created by the PR [1].
>
> As I see it, checking for changes and submitting the results should
> stay on the current host. However, it should be able to run all
> the actual work on slaves and retrieve the results from them. I would
> appreciate all the help with implementing that, plus possibly some
> degree of control to make it possible to update pkgcheck on slaves
> easily.
>
> Please let me know if you're interested in helping. Thanks.
>
> [1]:https://bitbucket.org/mgorny/repo-mirror-ci/src/
> master/pull-requests.job
>
> --
> Best regards,
> Michał Górny
>

[gentoo-dev] Request for help: distributed pull request CI (pkgcheck)

[Michał Górny]

Hello, everyone.

TL;DR: I'm looking for some help to get pull-request CI running
distributed, i.e. on multiple hosts.


Quick history
-

The repo-mirror-ci project pretty much started as a simple cronjob
running on hardware contributed to the purpose by Todd Goodman
(considering how much this hardware has done for Gentoo, I'd suggest
giving him a honorary citizenship of Gentoo or something like
that ;-)). I can't say they were terrible but they weren't perfect
either. Due to lack of time, I mostly focused on improving them
moderately and adding new features.

Today, everything is still running on the same hardware. While
the hardware is real good, and I'm doing some effort to improve
the performance of the tools, I think it's quite close to its limits.

Right now, it is running three tasks every 20 minutes: updating repo
mirrors, running pkgcheck on Gentoo and processing one pull request.
However, if those tasks delay beyond those 20 minutes, the poor man's
scheduling based on cron + lockfiles may cause some of the executions
to be skipped.

In other words, the best case is processing 3 pull requests / h. This
is starting to be no longer sufficient, and causing significant delays
during periods of high contributor activity -- while my goal would be to
provide the results ASAP -- 10-20 minutes would be perfect.


The goal and needs
--

I think the best way forward would be to start distributing tasks
between more systems. While I don't really have time to make whole
repo-mirror-ci distributed, I think it should be feasible to start
splitting pull request processing out of it.

I would use some help to achieve that, esp. wrt distributed task
scheduling.

Can you think of any tools that could help me get the task done easily
and with as little of reinventing the wheel as possible? Right now, I
have just a lot of trivial shell script that checks pull request for
changes, checks them out, runs 'pmaint regen', pkgcheck, publishes all
kinds of results and statuses, and also compares QA results to
determine new failures created by the PR [1].

As I see it, checking for changes and submitting the results should
stay on the current host. However, it should be able to run all
the actual work on slaves and retrieve the results from them. I would
appreciate all the help with implementing that, plus possibly some
degree of control to make it possible to update pkgcheck on slaves
easily.

Please let me know if you're interested in helping. Thanks.

[1]:https://bitbucket.org/mgorny/repo-mirror-ci/src/master/pull-requests.job

-- 
Best regards,
Michał Górny


pgpU8Dli2RywM.pgp
Description: OpenPGP digital signature