Cross-posting from the [email protected]. Answering Sander's request for guidance and help.
I am happy to provide help with my experience from Apache Airflow migrating to Github Actions. Side comment: I love the way GitHub actions evolve and I have quite an experience with running and optimizing our builds for Apache Airflow. We are really happy with it especially comparing to Travis CI we used previously. We've been waiting > 1 year (!!!) for GitLab to add running CI jobs for fork pull requests (https://gitlab.com/gitlab-org/gitlab/-/issues/5667 - the issue is already 2 years old and keeps on being pushed to the next release with every release so we stopped waiting for it. The speed with which Github Actions adds new features is impressive, to say the least, and their public roadmap https://github.com/github/roadmap/projects/1 is even more impressive). I am right now in the process of testing a big overhaul of the way how Apache Airflow uses Github Actions. Mostly it is about consistency and optimizations of waiting time + decreasing the pressure on the Github Actions workers we use (we have rather long and comprehensive tests and big matrix of jobs). You can see some (almost completed) work in progress here https://github.com/potiuk/airflow/tree/master/.github/workflows. Along the way I am also finishing up a Github Action that is going to make full use of recently released "workflow_run" type of triggers - that enabled automated canceling of duplicated Pull Request builds from forks, which was a major pain for us (and so far we only workaround it with cancel run scheduled every 5 minutes - which was pretty nasty :(. With the new "workflow_run" it seems possible, though a bit complex, that's why I took an existing cancel-workflow action written by someone else and added full support for workflow_runs (workflow_runs have been added ~ 1 week ago or so so they are rather fresh). The action is here: https://github.com/potiuk/cancel-workflow-runs#cancel-duplicate-jobs-for-triggered-workflow (the master version is what I am testing right now and will release v2 shortly with full support for workflow_run but the documentation is already rather comprehensive and contains examples and explains how things work. I am also helping the Apache Beam team in their Github Action migration, so happy to help you as well :). I am also working on handling flaky tests (and sharing experiences with the Beam team as well). We've introduced the concept of "Quarantined tests" and I am looking into using https://github.com/GoogleCloudPlatform/flaky-service - a service developed by Google Cloud Platform to manage flaky tests (It has Github Action integration out-of-the-box). You might also find useful the architecture of our integration between Github Actions and DockerHub and the approach we've taken so far (I contributed it to infra some time ago https://cwiki.apache.org/confluence/display/INFRA/Github+Actions+to+DockerHub). It will be slightly updated with the Overhaul I am working on, so I am going to update it shortly, the in-progress version of it is in https://github.com/potiuk/airflow/blob/master/CI.rst On 2020/08/13 15:06:18, Chandan Singh <[email protected]> wrote: > Hi Jürg, > > > Can't it import issues 1:1 without changing the issue numbering? I was > > expecting that based on `usePlaceholderIssuesForMissingIssues` as > > described in the README. This would also allow issue references in the > > git commit history to remain valid. > > > > This can't work for PRs as GitHub uses a shared namespace for issue and > > PR numbers while GitLab doesn't. However, I think issue references are > > more important than PR references. And I'm hoping that the GitLab MRs > > can stay up (as read-only archive). > > Sorry, I overlooked this. You are right - we can indeed preserve the > mapping between issues. Initially I only looked at some MR references, > and assumed that the same would apply to issues. > > But, I did run with `usePlaceholderIssuesForMissingIssues: true` and > that mapping seems to have been preserved. For example > https://github.com/cs-shadow/buildstream/issues/140#issuecomment-673461465. > > MR references still don't seem to be preserved though. (I can live with that.) > > - Chandan >
