This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch fix-edge-case-with-conflicting-deps in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 6a5ed98ec760fd0ede3415a25e8062b44d6f3d60 Author: Jarek Potiuk <[email protected]> AuthorDate: Mon Aug 28 21:26:31 2023 +0200 Fix edge case with constraints conflicting with setup dependencies There was an edge case with race condition between merging the setup.py change to main and refreshing constraint causing a failure of PROD image building when the new setup.py had conflicting dependencies with the past constraints. This happened on Aug 28 with azure-mgmt-containerinstance which got upgrade to a new 8.0.0 version which was conflicting with previous setup.py: * azure-mgmt-containerinstance>=1.5.0,<2.0 to * azure-mgmt-containerinstance>=7.0.0,<9.0.0 The problem was that PROD builds used the constraints from main to build the image, because constraints were not produced in the CI builds for the same job (constraints wer only produced in the PR that had "upgrade to newer depedencies". The regular PR CI builds have a build-in protection against such edge case - they run with `--upgrade-on-failure` flag that causes the PRs to upgrade their constraints eagerly if they fail on constraints failure. This PR fixes the edge case by utilising the result of the flag --upgrade-on failure. After this PR, constraints will always be uploaded by CI jobs, not only when "upgrade to newer dependencies" PR is run. this means that when CI build falls back as result of `--upgrade-on-failure` the newly generated constraints are uploaded as artifacts and used by PROD image build. As a side-effect, every PR will produce the constraints used by that build as artifact. This is actually pretty good thing for diagnosis of dependency problems. --- .github/actions/build-ci-images/action.yml | 4 +--- .github/actions/build-prod-images/action.yml | 1 - 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/.github/actions/build-ci-images/action.yml b/.github/actions/build-ci-images/action.yml index 415819008c..a622da980f 100644 --- a/.github/actions/build-ci-images/action.yml +++ b/.github/actions/build-ci-images/action.yml @@ -39,13 +39,12 @@ runs: run: > breeze release-management generate-constraints --run-in-parallel --airflow-constraints-mode constraints-source-providers - if: env.UPGRADE_TO_NEWER_DEPENDENCIES != 'false' - name: "Generate PyPI constraints" shell: bash run: > breeze release-management generate-constraints --run-in-parallel --airflow-constraints-mode constraints - if: env.UPGRADE_TO_NEWER_DEPENDENCIES != 'false' and ${{ inputs.build-provider-packages != 'true' }} + if: ${{ inputs.build-provider-packages != 'true' }} - name: "Print dependency upgrade summary" shell: bash run: | @@ -53,7 +52,6 @@ runs: echo "Summarizing Python $PYTHON_VERSION" cat "files/constraints-${PYTHON_VERSION}/*.md" >> $GITHUB_STEP_SUMMARY || true done - if: env.UPGRADE_TO_NEWER_DEPENDENCIES != 'false' - name: "Upload constraint artifacts" uses: actions/upload-artifact@v3 with: diff --git a/.github/actions/build-prod-images/action.yml b/.github/actions/build-prod-images/action.yml index 1223b5a7f1..0da3568680 100644 --- a/.github/actions/build-prod-images/action.yml +++ b/.github/actions/build-prod-images/action.yml @@ -61,7 +61,6 @@ runs: with: name: constraints path: ./docker-context-files - if: env.UPGRADE_TO_NEWER_DEPENDENCIES != 'false' - name: "Build & Push PROD images with source providers ${{ env.IMAGE_TAG }}:${{ env.PYTHON_VERSIONS }}" shell: bash run: >
