All merged, - we have now no "known" pull_request_target workflow in our "apache" branches - and we have "main", "v2-10-test" and "providers-fab/v1-5" branches working with new - simpler and more secure - workflows.
J. On Sat, Jan 11, 2025 at 8:25 PM Jarek Potiuk <ja...@potiuk.com> wrote: > A follow up here... > > Since the "pull_request_target" removal has to be done in all active > branches, I also backported the change to v2-10-test. This is a MASSIVE PR > - because rather than cherry-picking all changes from "main" I had to > simply take main version of our CI and dev scripts, copy them to v2-10-test > and adapt them back to 2.10 reality (providers in "airflow/providers", > Python 3.8, no "task_sdk" and so on). > https://github.com/apache/airflow/pull/45527 > > I proposed how this can be reviewed despite it being a massive change > (+13,819 −11,299) - but even that is somewhat revieweable. This is really a > "hybrid" case - as part of the code ("ci and dev scripts") is taken from > `main` and the "production code" is from `v2-10-test", if you make two > comparisons - for those two parts - each of them should make sense. > > I know I am asking a lot from reviewers, but I would love to finish it. > > Last step will also be backporting the change to `providers/fab-v1-5` > branch - but since that one was done after all the major changes above, > that should be easy. > > J. > > > > On Mon, Jan 6, 2025 at 4:46 PM Vincent Beck <vincb...@apache.org> wrote: > >> Wow! Massive change! Thanks to all who contributed :) >> >> On 2025/01/06 08:05:20 Ephraim Anierobi wrote: >> > Awesome work! Thank you, Jarek and Pavan! >> > >> > - Ephraim >> > >> > On Sat, 4 Jan 2025 at 13:45, Shahar Epstein <sha...@apache.org> wrote: >> > >> > > Well done Jarek and Pavan! >> > > >> > > Shahar >> > > >> > > On Mon, Dec 30, 2024 at 12:15 AM Jarek Potiuk <ja...@potiuk.com> >> wrote: >> > > >> > > > Hello here, >> > > > >> > > > TL;DR; I just merged https://github.com/apache/airflow/pull/45266 - >> > > > which implemented a much simplified and nicer workflow for our CI. >> > > > >> > > > Rebase to the latest `main` and you should be good to go. >> > > > >> > > > It (finally) switches o from a workflow we had for years (using >> pretty >> > > > dangerous from the security point of view `pull_request_target` >> > > workflow) - >> > > > into using Artifacts for sharing images in workflow. This was >> possible >> > > > thanks to new "artifacts" actions and switching to UV. >> > > > >> > > > The benefit of it is that it is way safer - no more "dangerous >> workflows" >> > > > and simpler - we have a lot simpler Dockerfile.ci and caching >> mechanism >> > > > implemented. We worked this out by discussing with other ASF >> projects and >> > > > actually even reusing an action developed by a fellow Apache Arrow >> > > > committer and PMC member - Jacob Wujciak. >> > > > >> > > > The things everyone should do: >> > > > >> > > > * rebase your PR to latest main to make your PRs rebuilt using the >> new >> > > > workflow >> > > > * run `breeze ci-image build` if you are using breeze locally >> > > > >> > > > I expect some teething problems, so do not hesitate to raise your >> > > problems >> > > > in #internal-airflow-ci-cd channel for CI or #airflow-breeze >> channel if >> > > you >> > > > see breeze problems >> > > > >> > > > Your regular workflows should continue working as usual, you should >> see >> > > > just one workflow in CI running builds and tests instead of two. >> > > > >> > > > J. >> > > > >> > > >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >> For additional commands, e-mail: dev-h...@airflow.apache.org >> >>
