bugraoz93 commented on code in PR #45324: URL: https://github.com/apache/airflow/pull/45324#discussion_r1900375845
########## dev/breeze/doc/ci/07_running_ci_locally.md: ########## @@ -79,83 +79,109 @@ To load the image from specific PR, you can use the following command: breeze ci-image load --from-pr 12345 --python 3.9 --github-token <your_github_token> ``` -To load the image from specific job run (for example 12538475388), +To load the image from specific run (for example 12538475388), you can use the following command, find the run id from github action runs. ```bash -breeze ci-image load --from-job 12538475388 --python 3.9 --github-token <your_github_token> +breeze ci-image load --from-run 12538475388 --python 3.9 --github-token <your_github_token> ``` After you load the image, you can reproduce the very exact environment that was used in the CI run by entering breeze container without mounting your local sources: ```bash -breeze shell --mount-sources skip [OTHER OPTIONS] +breeze shell --mount-sources skip [OPTIONS] ``` And you should be able to run any tests and commands interactively in the very exact environment that was used in the failing CI run even without checking out sources of the failing PR. This is a powerful tool to debug and fix CI issues. +You can also build the image locally by checking-out the branch of the PR that was used and running: + +```bash +breeze ci-image build +``` + +You have to be aware that some of the PRs and canary builds use the `--upgrade-to-newer-dependencies` flag +(`UPGRADE_TO_NEWER_DEPENDENCIES` environment variable set to `true`) and they are not using constraints +to build the image so if you want to build it locally, you should pass the `--upgrade-to-newer-dependencies` +flaag when you are building the image. + +Note however, that if constraints changed for regulare builds and if someone released a new package in PyPI +since the build was run (which is very likely - we have many packages released a day), the image you +build locally might be different than the one in CI, that's why loading image using `breeze ci-image load` +is more reliable way to reproduce the CI build. + If you check-out the branch of the PR that was used, regular ``breeze`` commands will also reproduce the CI environment without having to rebuild the image - for example when dependencies changed or when new dependencies were released and used in the CI job - and you will -be able to edit source files locally as usual. +be able to edit source files locally as usual and use your IDE and tools you usually use to develop Airflow. +In order to reproduce the exact job you also need to set the "[OPTIONS]" corresponding to the particular +job you want to reproduce within the run. You can find those in the logs of the CI job. Note that some +of the options can be passed by `--flags` and some via environment variables, for convenience, so you should +take a look at both if you want to be sure to reproduce the exact job configuration. See the next chapter +for summary of the most important environment variables and options used in the CI jobs. You can read more about it in [Breeze](../README.rst) and [Testing](../../../../contributing-docs/09_testing.rst) +# Options and environment variables used + Depending whether the scripts are run locally via [Breeze](../README.rst) or whether they are run in -`Build Images` or `Tests` workflows they can take different values. +`Build Images` or `Tests` workflows can behave differentlt. Review Comment: Tiny one :) ```suggestion `Build Images` or `Tests` workflows can behave differently. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
