Thanks for moving this forward! On a flip side, I noticed that many PreCommit actions (actually all for my PR #27851) are stuck waiting for a runner. Is it expected behaviour? Should we increase a number a runners while moving all Jenkins Jobs to GitHub actions?
— Alexey > On 13 Oct 2023, at 17:09, Yi Hu via dev <dev@beam.apache.org> wrote: > > Hi all, > > We have shutdown another set of Jenkins PreCommit, see the newly shutdown > ones in https://github.com/apache/beam/pull/28840 , And a complete list of > fully migrated tests tracked in https://github.com/apache/beam/issues/28426 > > Regards, > Yi > > On Thu, Sep 14, 2023 at 3:41 PM Yi Hu <ya...@google.com > <mailto:ya...@google.com>> wrote: >> Hi all, >> >> We just shutdown a second list of Jenkins test suites. The full list is >> tracked in https://github.com/apache/beam/issues/28426 . In the Issue there >> is also information about the status of migration as we proceed. >> >> Regards, >> Yi >> >> On Thu, Sep 14, 2023 at 12:44 PM Danny McCormick via dev >> <dev@beam.apache.org <mailto:dev@beam.apache.org>> wrote: >>> Sure, I added >>> https://cwiki.apache.org/confluence/display/BEAM/GitHub+Actions+Tips >>> >>> On Thu, Sep 14, 2023 at 12:39 PM Ahmet Altay <al...@google.com >>> <mailto:al...@google.com>> wrote: >>>> That is great. Could we add a link to that README from >>>> https://cwiki.apache.org/confluence/display/BEAM/Developer+Guides ? That >>>> will increase discoverability for people like me who use wiki as the >>>> starting point for finding how to do things. >>>> >>>> On Thu, Sep 14, 2023 at 8:41 AM Danny McCormick <dannymccorm...@google.com >>>> <mailto:dannymccorm...@google.com>> wrote: >>>>> Most of our docs for GitHub actions are located here - >>>>> https://github.com/apache/beam/blob/master/.github/workflows/README.md. I >>>>> added https://github.com/apache/beam/pull/28453 to add instructions to >>>>> that page. >>>>> >>>>> Thanks, >>>>> Danny >>>>> >>>>> On Wed, Sep 13, 2023 at 12:31 PM Ahmet Altay via dev <dev@beam.apache.org >>>>> <mailto:dev@beam.apache.org>> wrote: >>>>>> This is all great. Do you mind documenting the github actions flow for >>>>>> running these jobs on the wiki? Or if it already exists, share a link. >>>>>> Thank you! >>>>>> >>>>>> On Wed, Sep 13, 2023 at 6:19 AM XQ Hu via dev <dev@beam.apache.org >>>>>> <mailto:dev@beam.apache.org>> wrote: >>>>>>> This is awesome! Thanks, Danny, Yi, Andrey, and Vlado! >>>>>>> >>>>>>> On Wed, Sep 13, 2023 at 8:08 AM Danny McCormick via dev >>>>>>> <dev@beam.apache.org <mailto:dev@beam.apache.org>> wrote: >>>>>>>> Right now, it is just the set of jobs in Yi's PR - >>>>>>>> https://github.com/apache/beam/pull/28316/files but the plan is to >>>>>>>> slowly move jobs over time once we've built confidence that they work. >>>>>>>> >>>>>>>> There is a set of jobs that are non-idempotent which we are moving in >>>>>>>> one shot (which does include the website publishing job in this PR >>>>>>>> <https://github.com/apache/beam/pull/28391> that I just merged) and >>>>>>>> monitoring closely. The best source of truth is just looking at >>>>>>>> https://ci-beam.apache.org/ though. Any migrated jobs will have the >>>>>>>> disabled symbol next to them - for example, the whitespace jobs have >>>>>>>> now been migrated so they are disabled on Jenkins: >>>>>>>> >>>>>>>> <image.png> >>>>>>>> >>>>>>>> To run a job that has been migrated manually, you can navigate to that >>>>>>>> job in the Actions tab and click "run workflow". So for the website >>>>>>>> publish job you would navigate to >>>>>>>> https://github.com/apache/beam/actions/workflows/beam_PostCommit_Website_Publish.yml >>>>>>>> and click this button: >>>>>>>> >>>>>>>> >>>>>>>> <image.png> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Danny >>>>>>>> >>>>>>>> On Tue, Sep 12, 2023 at 6:22 PM Ahmet Altay via dev >>>>>>>> <dev@beam.apache.org <mailto:dev@beam.apache.org>> wrote: >>>>>>>>> Thank you for doing this. >>>>>>>>> >>>>>>>>> Is there a list of jobs that will be disabled? I am particularly >>>>>>>>> curious about: website publishing job (which I need to use manually >>>>>>>>> sometimes) and the job that publishes daily staging builds (which we >>>>>>>>> share with users sometimes.) >>>>>>>>> >>>>>>>>> Thank you. >>>>>>>>> Ahmet >>>>>>>>> >>>>>>>>> On Tue, Sep 12, 2023 at 11:14 AM Danny McCormick via dev >>>>>>>>> <dev@beam.apache.org <mailto:dev@beam.apache.org>> wrote: >>>>>>>>>> Hey everyone, I wanted to let you know that as part of the migration >>>>>>>>>> from Jenkins to GitHub Actions we are going to start disabling >>>>>>>>>> Jenkins jobs if they have a corresponding GitHub Actions job that >>>>>>>>>> has been running successfully for a while. We are starting with Yi's >>>>>>>>>> PR here - https://github.com/apache/beam/pull/28316. This is the >>>>>>>>>> next step in the process we kicked off last year [1] now that >>>>>>>>>> self-hosted runners have been in place and working for a while [2]. >>>>>>>>>> >>>>>>>>>> We will not migrate jobs until we've confirmed we have parity with >>>>>>>>>> the existing Jenkins implementations (for example, some jobs are >>>>>>>>>> still missing test publishing and we won't remove the Jenkins >>>>>>>>>> version until they have it). In the meantime, migrating some load >>>>>>>>>> off should help reduce the overall load on Jenkins so that it >>>>>>>>>> experiences fewer issues. >>>>>>>>>> >>>>>>>>>> If you have any objections with this approach, please respond here. >>>>>>>>>> If you run into any problems, please file an issue and tag me >>>>>>>>>> (@damccorm), Yi (@abacn), Andrey (@andreydevyatkin), or Vlado >>>>>>>>>> (@volatilemolotov) - or just tag all of us :). >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Danny >>>>>>>>>> >>>>>>>>>> [1] https://lists.apache.org/thread/0brbkmbd522d1ow43gx5b13dmywt2dgn >>>>>>>>>> [2] - >>>>>>>>>> https://lists.apache.org/thread/3k1owt5k16byv39b9lszd3l7qv7od4r8
