The GitHub Actions job "Tests" on airflow.git has succeeded. Run started by GitHub user potiuk (triggered by potiuk).
Head commit for run: 547cd12346195dc1b1d8dda551f0f8d3c6d3c17e / Jarek Potiuk <[email protected]> Move prepare-provider-packages to breeze This PR moves the functionality of preparing provider packages from a python script inside Breeze CI image to breeze Python package. This is the first of the series of moves that will simplify the way how provider packages are being built and prepared with the aim of improving security of the supply chain and make it easier to debug and modify the release process. Historically, release process has been run inside of Breeze for several reasons: isolation of running package preparation from Host environment, the need to keep separate virtualenv and because we run verification of provider packages during release process - which requires the CI environment with all its dependencies. So far the process looked like this: * bash breeze parsed the arguments * bash breeze started the docker bash script with packages as parameters * the bash script in CI image looped over the packages and run python prepare_provider_packages.py (twice) to generate docs and update changelog (this is interactive process where release manager makes decision on bumping versions). Those python script performed verification on provider.yaml files * the bash script summarized the packages and displayed status of preparation However after moving to Python based breeze, we can simplify it all and run all those steps in Python breeze internal code - no need to go to docker and use bash scripts. We also do not have to do verification of provider.yaml files is done already extensively in pre-commit. This PR moves all this logic to inside Breeze. There is stil remainig duplicated code in the original in-container `prepare_provider_packages.py` remaining, this duplication will be removed by subsequent PRs where other release management commands for provider packages will also be moved to Breeze as follow-up of this PR. This PR has the following changes: * move the provider decumentation code from `dev/provider_packages` to `dev/breeze/` (and from in-container to in-breeze-venv execution) * completely removed the intermediate bash script and calling Python scripts from it - moving the logic to Breeze entirely * added better diagnostics of what happens when packages are classified with particular types of changes (added `special` style to show it) * cleaned and clarified `prepare-provider-documentation` commmand line flags * introduce explicit "non-interactive" mode that is used to run and test the command in CI and to test it locally * replace str with Path were files were used in the moved code * add unit tests covering unit-testable parts of the moved code * refactore the moved code to use utils available in Breeze * split the code into packages and versions (reusable utils) and specific code for preparing package documentation * cached provider.yaml information retrieved from providers * move provider documentation templates to Breeze * better error handling - they are now regular exceptions in Python process rather than interpreting the exit_codes passed from python sub-scripts returnig the codes to Bash * when release manager classifies package, only relevant sections are generated (Features/Breaking changes) based on decision and changes are automatically "guessed" only if release manager chose the section where they would fall in Report URL: https://github.com/apache/airflow/actions/runs/6842738074 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
