The number of runners should autoscale, but it seems that they did not do so quickly enough in this instance. For the short term, we have set a higher minimum number of runners (and are currently all caught up on backlog) and we will investigate the underlying issue.
Thanks for calling this out Alexey! On Mon, Oct 16, 2023 at 10:48 AM Alexey Romanenko <aromanenko....@gmail.com> wrote: > Thanks for moving this forward! > > On a flip side, I noticed that many PreCommit actions (actually all for my > PR #27851) are stuck waiting for a runner. > Is it expected behaviour? > Should we increase a number a runners while moving all Jenkins Jobs to > GitHub actions? > > — > Alexey > > On 13 Oct 2023, at 17:09, Yi Hu via dev <dev@beam.apache.org> wrote: > > Hi all, > > We have shutdown another set of Jenkins PreCommit, see the newly shutdown > ones in https://github.com/apache/beam/pull/28840 , And a complete list > of fully migrated tests tracked in > https://github.com/apache/beam/issues/28426 > > Regards, > Yi > > On Thu, Sep 14, 2023 at 3:41 PM Yi Hu <ya...@google.com> wrote: > >> Hi all, >> >> We just shutdown a second list of Jenkins test suites. The full list is >> tracked in https://github.com/apache/beam/issues/28426 . In the Issue >> there is also information about the status of migration as we proceed. >> >> Regards, >> Yi >> >> On Thu, Sep 14, 2023 at 12:44 PM Danny McCormick via dev < >> dev@beam.apache.org> wrote: >> >>> Sure, I added >>> https://cwiki.apache.org/confluence/display/BEAM/GitHub+Actions+Tips >>> >>> On Thu, Sep 14, 2023 at 12:39 PM Ahmet Altay <al...@google.com> wrote: >>> >>>> That is great. Could we add a link to that README from >>>> https://cwiki.apache.org/confluence/display/BEAM/Developer+Guides ? >>>> That will increase discoverability for people like me who use wiki as the >>>> starting point for finding how to do things. >>>> >>>> On Thu, Sep 14, 2023 at 8:41 AM Danny McCormick < >>>> dannymccorm...@google.com> wrote: >>>> >>>>> Most of our docs for GitHub actions are located here - >>>>> https://github.com/apache/beam/blob/master/.github/workflows/README.md. >>>>> I added https://github.com/apache/beam/pull/28453 to add instructions >>>>> to that page. >>>>> >>>>> Thanks, >>>>> Danny >>>>> >>>>> On Wed, Sep 13, 2023 at 12:31 PM Ahmet Altay via dev < >>>>> dev@beam.apache.org> wrote: >>>>> >>>>>> This is all great. Do you mind documenting the github actions flow >>>>>> for running these jobs on the wiki? Or if it already exists, share a >>>>>> link. >>>>>> Thank you! >>>>>> >>>>>> On Wed, Sep 13, 2023 at 6:19 AM XQ Hu via dev <dev@beam.apache.org> >>>>>> wrote: >>>>>> >>>>>>> This is awesome! Thanks, Danny, Yi, Andrey, and Vlado! >>>>>>> >>>>>>> On Wed, Sep 13, 2023 at 8:08 AM Danny McCormick via dev < >>>>>>> dev@beam.apache.org> wrote: >>>>>>> >>>>>>>> Right now, it is just the set of jobs in Yi's PR - >>>>>>>> https://github.com/apache/beam/pull/28316/files but the plan is to >>>>>>>> slowly move jobs over time once we've built confidence that they work. >>>>>>>> >>>>>>>> There is a set of jobs that are non-idempotent which we are moving >>>>>>>> in one shot (which does include the website publishing job in this >>>>>>>> PR <https://github.com/apache/beam/pull/28391> that I just merged) >>>>>>>> and monitoring closely. The best source of truth is just looking at >>>>>>>> https://ci-beam.apache.org/ though. Any migrated jobs will have >>>>>>>> the disabled symbol next to them - for example, the whitespace jobs >>>>>>>> have >>>>>>>> now been migrated so they are disabled on Jenkins: >>>>>>>> >>>>>>>> <image.png> >>>>>>>> >>>>>>>> To run a job that has been migrated manually, you can navigate to >>>>>>>> that job in the Actions tab and click "run workflow". So for the >>>>>>>> website >>>>>>>> publish job you would navigate to >>>>>>>> https://github.com/apache/beam/actions/workflows/beam_PostCommit_Website_Publish.yml >>>>>>>> and >>>>>>>> click this button: >>>>>>>> >>>>>>>> >>>>>>>> <image.png> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Danny >>>>>>>> >>>>>>>> On Tue, Sep 12, 2023 at 6:22 PM Ahmet Altay via dev < >>>>>>>> dev@beam.apache.org> wrote: >>>>>>>> >>>>>>>>> Thank you for doing this. >>>>>>>>> >>>>>>>>> Is there a list of jobs that will be disabled? I am particularly >>>>>>>>> curious about: website publishing job (which I need to use manually >>>>>>>>> sometimes) and the job that publishes daily staging builds (which we >>>>>>>>> share >>>>>>>>> with users sometimes.) >>>>>>>>> >>>>>>>>> Thank you. >>>>>>>>> Ahmet >>>>>>>>> >>>>>>>>> On Tue, Sep 12, 2023 at 11:14 AM Danny McCormick via dev < >>>>>>>>> dev@beam.apache.org> wrote: >>>>>>>>> >>>>>>>>>> Hey everyone, I wanted to let you know that as part of the >>>>>>>>>> migration from Jenkins to GitHub Actions we are going to start >>>>>>>>>> disabling >>>>>>>>>> Jenkins jobs if they have a corresponding GitHub Actions job that >>>>>>>>>> has been >>>>>>>>>> running successfully for a while. We are starting with Yi's PR here - >>>>>>>>>> https://github.com/apache/beam/pull/28316. This is the next step >>>>>>>>>> in the process we kicked off last year [1] now that self-hosted >>>>>>>>>> runners >>>>>>>>>> have been in place and working for a while [2]. >>>>>>>>>> >>>>>>>>>> We will not migrate jobs until we've confirmed we have parity >>>>>>>>>> with the existing Jenkins implementations (for example, some jobs >>>>>>>>>> are still >>>>>>>>>> missing test publishing and we won't remove the Jenkins version >>>>>>>>>> until they >>>>>>>>>> have it). In the meantime, migrating some load off should help >>>>>>>>>> reduce the >>>>>>>>>> overall load on Jenkins so that it experiences fewer issues. >>>>>>>>>> >>>>>>>>>> If you have any objections with this approach, please respond >>>>>>>>>> here. If you run into any problems, please file an issue and tag me >>>>>>>>>> (@damccorm), Yi (@abacn), Andrey (@andreydevyatkin), or Vlado >>>>>>>>>> (@volatilemolotov) - or just tag all of us :). >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Danny >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> https://lists.apache.org/thread/0brbkmbd522d1ow43gx5b13dmywt2dgn >>>>>>>>>> [2] - >>>>>>>>>> https://lists.apache.org/thread/3k1owt5k16byv39b9lszd3l7qv7od4r8 >>>>>>>>>> >>>>>>>>> >
