This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch v3-4-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/v3-4-test by this push:
new a187d8ac2b0 [v3-4-test] Run dependency explanations separately from
main AMD canaries (#74311) (#76182)
a187d8ac2b0 is described below
commit a187d8ac2b0a65a10336ee944bf6878b292d3b3c
Author: github-actions[bot]
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Sat Oct 31 22:02:24 2026 +0100
[v3-4-test] Run dependency explanations separately from main AMD canaries
(#74311) (#76182)
* Keep dependency upgrade explanations off the canary completion path
Detailed upgrade investigations grow with the number of outdated packages
and delay canary completion even when test preparation gets faster. Maintainers
need prompt test results while retaining reproducible upgrade diagnostics.
* Keep dependency reports from writing to the main branch cache
Diagnostic reruns must retain their original inputs without gaining cache
write access on the default branch. The workflow-run event already provides
that boundary and the standard rerun action preserves the source canary.
* Keep dependency reports aligned with supported Python versions
* Keep dependency reports going when one input is gone
- Skip entries whose saved constraints or CI image are missing or expired,
with a warning and a summary line, instead of failing every report.
- Name report inputs by canary attempt and explain each input once, so a
re-run canary no longer queues a second report from the same inputs.
- Post the same notice on every attempt when the canary saved no inputs.
- Free up disk space and move Docker to /mnt before loading the image.
- Save the checked constraints through breeze's --save-constraints-to,
pinned to the resolved constraints commit; failing to save only warns,
so the canary no longer depends on raw `gh api` calls.
- Say that no cache step, not workflow_run, keeps the cache untouched, and
skip the plan tests when node is unavailable.
* Give dependency explanations room for the slowest constraints mode
Explaining constraints-source-providers took 105-120 minutes per job in
the last four main canaries, past the report job's 90-minute timeout.
* Honor CONSTRAINTS_GITHUB_REPOSITORY when breeze downloads constraints
CONSTRAINTS_GITHUB_REPOSITORY names the repository holding the
constraints-* branches, apache/airflow by default, and the Dockerfiles
read it. Breeze's download_constraints_file and get_branch_commit always
used apache/airflow instead, so setting it changed nothing for
constraints-version-check, the SBOM and provider dependency downloads,
or the constraints commit saved for the dependency report.
Both functions now read CONSTRAINTS_GITHUB_REPOSITORY and fall back to
apache/airflow when it is unset or empty, so existing callers and CI on
apache/airflow behave as before, and a saved commit always comes from
the repository its constraints were downloaded from. Other files fetched
through download_file_from_github still come from apache/airflow.
* Explain dependencies from the image a RunsOn canary keeps in S3
Since #74889 scheduled canaries run on RunsOn, where the CI image stash
goes to RunsOn's S3 cache and only its digest is a GitHub artifact, so
every report entry was skipped as "CI image missing". The plan now also
accepts that digest artifact; the explain job then runs on a RunsOn
runner, downloads exactly that tarball by its digest and checks it like
stash-restore does. It prepares the disk with the workflow version's
runner scripts, as the image build job now does.
* Use the current RunsOn runner label for dependency reports from S3
main changed breeze's RUNS_ON_AMD_RUNNER_TEMPLATE (instance families
and spot strategy); the report's copy now matches it again, as the plan
test that compares the two requires.
* Describe re-used report inputs as saved and refuse parent segments in the
S3 stash
An earlier canary attempt only saved the inputs; its report may have
failed, so the summary should not call them explained. The stash tarball is
extracted with tar -P, so an entry with a .. segment could leave /mnt even
under the /mnt prefix check.
Generated-by: Claude Opus 5
---------
(cherry picked from commit ca6861474f189f5ac26d5cecaad3300a12706375)
Co-authored-by: Yossi Eliaz <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Shahar Epstein <[email protected]>
Co-authored-by: Jarek Potiuk <[email protected]>
---
.github/workflows/dependency-upgrade-report.yml | 254 +++++++++++++
.github/workflows/finalize-tests.yml | 29 +-
dev/breeze/doc/ci/05_workflows.md | 45 +++
...elease-management_constraints-version-check.svg | 60 ++--
...elease-management_constraints-version-check.txt | 2 +-
.../commands/release_management_commands.py | 20 ++
.../commands/release_management_commands_config.py | 2 +
.../utils/constraints_version_check.py | 67 +++-
dev/breeze/src/airflow_breeze/utils/github.py | 58 ++-
dev/breeze/tests/test_constraints_version_check.py | 140 ++++++++
dev/breeze/tests/test_dependency_upgrade_report.py | 396 +++++++++++++++++++++
dev/breeze/tests/test_github_utils.py | 81 +++++
12 files changed, 1104 insertions(+), 50 deletions(-)
diff --git a/.github/workflows/dependency-upgrade-report.yml
b/.github/workflows/dependency-upgrade-report.yml
new file mode 100644
index 00000000000..df9c7e33f85
--- /dev/null
+++ b/.github/workflows/dependency-upgrade-report.yml
@@ -0,0 +1,254 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+name: Dependency upgrade report
+# Both the job gate and the API validation reject PRs and foreign
repositories. Only
+# completed main schedule/dispatch runs are executed, with a read-only token
and no secrets.
+"on": # zizmor: ignore[dangerous-triggers]
+ workflow_run:
+ workflows: ["Tests (AMD)"]
+ types: [completed]
+ branches: [main]
+permissions:
+ contents: read
+ actions: read
+# Every canary attempt fires this workflow and gets its own report of the
inputs it saved.
+concurrency:
+ group: dependency-report-${{ github.event.workflow_run.id }}-${{
github.event.workflow_run.run_attempt }}
+ cancel-in-progress: false
+jobs:
+ plan:
+ runs-on: ubuntu-slim
+ timeout-minutes: 5
+ if: >-
+ github.event.workflow_run.head_repository.full_name == github.repository
&&
+ (github.event.workflow_run.event == 'schedule' ||
+ github.event.workflow_run.event == 'workflow_dispatch')
+ outputs:
+ matrix: ${{ steps.plan.outputs.matrix }}
+ has-reports: ${{ steps.plan.outputs.has-reports }}
+ source-sha: ${{ steps.plan.outputs.source-sha }}
+ source-run-id: ${{ steps.plan.outputs.source-run-id }}
+ steps:
+ - name: "Select inputs from the original canary run"
+ id: plan
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3
# v9.0.0
+ env:
+ SOURCE_RUN_ID: ${{ github.event.workflow_run.id }}
+ SOURCE_RUN_ATTEMPT: ${{ github.event.workflow_run.run_attempt }}
+ with:
+ script: |
+ const rawId = process.env.SOURCE_RUN_ID;
+ if (!/^[1-9][0-9]*$/.test(rawId)) throw new Error('Invalid source
run ID');
+ const run_id = Number(rawId);
+ if (!Number.isSafeInteger(run_id)) throw new Error('Invalid source
run ID');
+ const rawAttempt = process.env.SOURCE_RUN_ATTEMPT;
+ if (!/^[1-9][0-9]{0,3}$/.test(rawAttempt)) throw new
Error('Invalid source run attempt');
+ const attempt_number = Number(rawAttempt);
+ const repo = context.repo;
+ const { data: run } = await
github.rest.actions.getWorkflowRunAttempt(
+ { ...repo, run_id, attempt_number });
+ if (run.head_repository.full_name !== `${repo.owner}/${repo.repo}`
||
+ run.head_branch !== 'main' || run.path !==
'.github/workflows/ci-amd.yml' ||
+ !['schedule', 'workflow_dispatch'].includes(run.event) ||
run.status !== 'completed') {
+ throw new Error('Only completed main AMD canary runs are
supported');
+ }
+ const artifacts = await
github.paginate(github.rest.actions.listWorkflowRunArtifacts,
+ { ...repo, run_id, per_page: 100 });
+ const include = [];
+ const skipped = [];
+ const explained = [];
+ const inputName = new
RegExp('^dependency-report-inputs-(3\\.[0-9]+)-' +
+
'(constraints(?:-source-providers|-no-providers)?)-(true|false)-([1-9][0-9]*)$');
+ const saved = artifacts.map(a => [a,
inputName.exec(a.name)]).filter(([, match]) => match);
+ // Only RunsOn runners can read RunsOn's S3 cache, where a canary
on RunsOn keeps its CI image.
+ const runsOnRunner =
`runs-on=${process.env.GITHUB_RUN_ID}/family=m8a+m8i+m7a+m7i+m6i` +
+ '/image=ubuntu22-full-x64/volume=150gb/spot=co/cpu=8/ram=32';
+ for (const [input, [, python, mode, useUv, attempt]] of saved) {
+ if (attempt !== rawAttempt) continue;
+ const entry = `${python}:${mode}`;
+ // A re-run canary saves its inputs again; they are explained
only for the first attempt.
+ if (saved.some(([, m]) => m[1] === python && m[2] === mode &&
m[3] === useUv &&
+ Number(m[4]) < attempt_number)) {
+ explained.push(entry);
+ continue;
+ }
+ // The stash action names the artifact after the stash key, with
'/' as '_', and the branch.
+ // On RunsOn runners the image itself goes to RunsOn's S3 cache
and the artifact is only
+ // its digest, stashed under the key plus '-s3' (see
.github/actions/stash-save).
+ const imageName = `ci-image-save-v3-linux_amd64-${python}-main`;
+ const image = artifacts.find(a => a.name === imageName) ||
+ artifacts.find(a => a.name ===
`ci-image-save-v3-linux_amd64-${python}-s3-main`);
+ const missing = input.expired ? 'inputs expired' :
+ !image ? 'CI image missing' : image.expired ? 'CI image
expired' : '';
+ if (missing) {
+ core.warning(`Skipping ${entry}: original inputs or CI image
unavailable (${missing})`);
+ skipped.push(`${entry} (${missing})`);
+ continue;
+ }
+ const fromS3 = image.name !== imageName;
+ include.push({ python, mode, 'use-uv': useUv, 'inputs-id':
input.id, 'image-id': image.id,
+ 'image-source': fromS3 ? 's3' : 'artifact', runner: fromS3 ?
runsOnRunner : 'ubuntu-22.04' });
+ }
+ core.setOutput('matrix', JSON.stringify({ include }));
+ core.setOutput('has-reports', include.length > 0 ? 'true' :
'false');
+ core.setOutput('source-sha', run.head_sha);
+ core.setOutput('source-run-id', rawId);
+ if (!include.length && skipped.length &&
Number(process.env.GITHUB_RUN_ATTEMPT) > 1) {
+ throw new Error('Saved report inputs are missing or expired');
+ }
+ if (!include.length && !skipped.length) {
+ core.notice(explained.length ?
+ 'The saved report inputs were already saved by an earlier
canary attempt' :
+ 'No saved report inputs: this run did not request independent
diagnostics');
+ }
+ await core.summary.addHeading('Dependency upgrade diagnostics')
+ .addLink(`Source canary run (attempt ${rawAttempt})`,
run.html_url)
+ .addRaw(`\n\nSource revision: ${run.head_sha}\n\nReports:
${include.length}\n`)
+ .addRaw(skipped.length ? `\nSkipped: ${skipped.join(', ')}\n` :
'')
+ .addRaw(explained.length ? `\nSaved by an earlier attempt:
${explained.join(', ')}\n` : '')
+ .write();
+
+ explain:
+ needs: plan
+ if: needs.plan.outputs.has-reports == 'true'
+ name: "Explain ${{ matrix.python }}:${{ matrix.mode }}"
+ runs-on: ${{ matrix.runner }}
+ # Explaining constraints-source-providers took up to 115 minutes in the
canary (Oct 2026).
+ timeout-minutes: 150
+ strategy:
+ fail-fast: false
+ max-parallel: 3
+ matrix: ${{ fromJSON(needs.plan.outputs.matrix) }}
+ env:
+ GITHUB_REPOSITORY: ${{ github.repository }}
+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ GITHUB_USERNAME: ${{ github.actor }}
+ VERBOSE: "false"
+ USE_UV: ${{ matrix.use-uv }}
+ steps:
+ - name: "Checkout the tested revision"
+ uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 #
v7.0.1
+ with:
+ ref: ${{ needs.plan.outputs.source-sha }}
+ persist-credentials: false
+ # The disk is prepared as in the image build jobs: the loaded image and
uv resolutions outgrow /,
+ # and the runner scripts come from the workflow's own version, which
knows RunsOn's disks.
+ - name: "Checkout the workflow version for its actions and runner
scripts"
+ uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 #
v7.0.1
+ with:
+ persist-credentials: false
+ path: workflow-version
+ sparse-checkout: |
+ .github/actions
+ scripts/ci
+ - name: "Free up disk space"
+ shell: bash
+ run: ./scripts/tools/free_up_disk_space.sh
+ - name: "Make /mnt writeable"
+ run: ./workflow-version/scripts/ci/make_mnt_writeable.sh
+ - name: "Move docker to /mnt"
+ run: ./workflow-version/scripts/ci/move_docker_to_mnt.sh
+ - name: "Install Breeze"
+ uses: ./.github/actions/breeze
+ - name: "Download the original CI image"
+ uses:
actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
+ with:
+ artifact-ids: ${{ matrix.image-id }}
+ run-id: ${{ needs.plan.outputs.source-run-id }}
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ merge-multiple: true
+ path: /mnt
+ if: matrix.image-source == 'artifact'
+ - name: "Download the digest of the original CI image"
+ uses:
actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
+ with:
+ artifact-ids: ${{ matrix.image-id }}
+ run-id: ${{ needs.plan.outputs.source-run-id }}
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ merge-multiple: true
+ path: ${{ runner.temp }}/ci-image-digest
+ if: matrix.image-source == 's3'
+ # The tarball is the one the canary stashed, addressed by its digest as
in
+ # .github/actions/stash-restore, so a later build on `main` cannot
replace it.
+ - name: "Restore the original CI image from the RunsOn cache"
+ shell: bash
+ run: |
+ set -euo pipefail
+ digest="$(cat "${RUNNER_TEMP}/ci-image-digest/sha256")"
+ if [[ ! "${digest}" =~ ^[0-9a-f]{64}$ ]]; then
+ echo "::error::Malformed digest of the original CI image"
+ exit 1
+ fi
+ export AWS_DEFAULT_REGION="${RUNS_ON_AWS_REGION:?not running on a
RunsOn runner}"
+ bucket="${RUNS_ON_S3_BUCKET_CACHE:?not running on a RunsOn runner}"
+ object="s3://${bucket}/cache/airflow-stash/sha256/${digest}.tar"
+ payload="${RUNNER_TEMP}/ci-image-stash.tar"
+ aws s3 cp --only-show-errors "${object}" "${payload}"
+ echo "${digest} ${payload}" | sha256sum --check --status
+ tar -tPf "${payload}" | while IFS= read -r entry; do
+ if [[ "${entry}" != /mnt/* || "${entry}" == */../* || "${entry}"
== */.. ]]; then
+ echo "::error::The original CI image stash holds ${entry}
outside /mnt"
+ exit 1
+ fi
+ done
+ tar -xPf "${payload}"
+ rm -f "${payload}"
+ if: matrix.image-source == 's3'
+ - name: "Load the original CI image"
+ env:
+ PYTHON_VERSION: ${{ matrix.python }}
+ run: breeze ci-image load --platform linux/amd64 --python
"${PYTHON_VERSION}" --image-file-dir /mnt
+ - name: "Download the constraints used by the quick summary"
+ uses:
actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
+ with:
+ artifact-ids: ${{ matrix.inputs-id }}
+ run-id: ${{ needs.plan.outputs.source-run-id }}
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ merge-multiple: true
+ path: files/dependency-report
+ - name: "Explain outdated dependencies"
+ shell: bash
+ env:
+ PYTHON_VERSION: ${{ matrix.python }}
+ CONSTRAINTS_MODE: ${{ matrix.mode }}
+ SOURCE_RUN_ID: ${{ needs.plan.outputs.source-run-id }}
+ SOURCE_SHA: ${{ needs.plan.outputs.source-sha }}
+ run: |
+ {
+ echo "## Dependency upgrade explanations"
+ echo "Source canary run: ${SOURCE_RUN_ID}; source revision:
${SOURCE_SHA}"
+ echo "Constraints commit: $(cat
files/dependency-report/constraints-commit.txt)"
+ echo "Python: ${PYTHON_VERSION}; mode: ${CONSTRAINTS_MODE}"
+ echo "The constraints are the constraints branch the canary's
summary checked, not ones built"
+ echo "from the source revision: dependency bounds changed between
the two can show up below as"
+ echo "conflicts or as stale constraints."
+ } >> "${GITHUB_STEP_SUMMARY}"
+ breeze release-management constraints-version-check \
+ --python "${PYTHON_VERSION}" --airflow-constraints-mode
"${CONSTRAINTS_MODE}" \
+ --constraints-file files/dependency-report/constraints.txt
--explain-why \
+ 2>&1 | tee files/dependency-report/explanations.txt
+ - name: "Save diagnostic output, including partial output on failure"
+ if: always()
+ uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9
# v7.0.2
+ with:
+ name: dependency-explanations-${{ matrix.python }}-${{ matrix.mode }}
+ path: files/dependency-report/
+ if-no-files-found: ignore
+ retention-days: 7
+ overwrite: true
diff --git a/.github/workflows/finalize-tests.yml
b/.github/workflows/finalize-tests.yml
index d9d5c463b15..262cc2b9a66 100644
--- a/.github/workflows/finalize-tests.yml
+++ b/.github/workflows/finalize-tests.yml
@@ -153,16 +153,37 @@ jobs:
python: ${{ matrix.python-version }}
use-uv: ${{ inputs.use-uv }}
make-mnt-writeable-and-cleanup: true
+ # Scheduled and dispatched canaries on main leave the explanations to
dependency-upgrade-report.yml
+ # and save the constraints they checked for it. Saving them never fails
this step.
- name: "Deps: ${{ matrix.python-version }}:${{ matrix.constraints-mode
}}"
shell: bash
- run: >
- breeze release-management constraints-version-check
- --python "${MATRIX_PYTHON_VERSION}"
- --airflow-constraints-mode "${MATRIX_CONSTRAINTS_MODE}" --explain-why
+ run: |
+ REPORT_ARGS=(--explain-why)
+ if [[ "${INDEPENDENT_REPORT}" == "true" ]]; then
+ REPORT_ARGS=(--no-explain-why --save-constraints-to
files/dependency-report)
+ fi
+ breeze release-management constraints-version-check \
+ --python "${MATRIX_PYTHON_VERSION}" \
+ --airflow-constraints-mode "${MATRIX_CONSTRAINTS_MODE}"
"${REPORT_ARGS[@]}"
env:
MATRIX_PYTHON_VERSION: "${{ matrix.python-version }}"
MATRIX_CONSTRAINTS_MODE: "${{ matrix.constraints-mode }}"
+ INDEPENDENT_REPORT: >-
+ ${{ github.ref == 'refs/heads/main' &&
+ (github.event_name == 'schedule' || github.event_name ==
'workflow_dispatch') }}
VERBOSE: "false"
+ - name: "Upload dependency report inputs"
+ if: >-
+ !cancelled() &&
hashFiles('files/dependency-report/constraints-commit.txt') != ''
+ continue-on-error: true
+ uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9
# v7.0.2
+ with:
+ name: "dependency-report-inputs-${{ matrix.python-version }}-${{
matrix.constraints-mode }}\
+ -${{ inputs.use-uv }}-${{ github.run_attempt }}"
+ path: files/dependency-report/
+ if-no-files-found: error
+ retention-days: 2
+ overwrite: true
push-buildx-cache-to-github-registry:
name: Push Regular Image Cache ${{ inputs.platform }}
diff --git a/dev/breeze/doc/ci/05_workflows.md
b/dev/breeze/doc/ci/05_workflows.md
index 0bbc0ffe5bb..a371d363004 100644
--- a/dev/breeze/doc/ci/05_workflows.md
+++ b/dev/breeze/doc/ci/05_workflows.md
@@ -563,6 +563,51 @@ unprivileged, which constrains what can go on it:
When adding a job, reach for `ubuntu-slim` if it only shuffles metadata
around, and
`ubuntu-22.04` otherwise.
+### Dependency upgrade diagnostics
+
+Scheduled and manually dispatched AMD canaries on `main` keep the dependency
freshness
+summary in `finalize-tests.yml`, but run it with `--no-explain-why`. Tests,
constraint
+generation and validation, and image publication still run in the canary.
Release-branch
+and other run types retain their existing explanation behavior. The summary
saves the
+constraints it checked, and the constraints-branch commit they came from, with
+`--save-constraints-to`; a failure to save them only warns and never fails the
canary.
+
+After the canary completes, `dependency-upgrade-report.yml` runs the detailed
explanations
+independently, from the inputs the canary saved; a canary with failed jobs
saves none. It checks out
+the original source SHA and downloads the original image and saved constraints
by artifact
+ID from that run. It never substitutes the latest branch image or constraints:
an entry whose
+saved constraints or image are missing or expired is skipped with a warning
and listed in the
+summary, and the others are still explained. The image comes from the CI image
stash, which
+keeps only the newest image per Python version and branch, so a later build on
`main` can
+replace it within hours. A canary on RunsOn keeps the image in RunsOn's S3
cache and stashes
+only its digest; the report job then runs on a RunsOn runner and downloads
exactly that tarball
+by its digest. Re-running the canary triggers the workflow again; inputs that
an earlier attempt
+already saved are not explained again.
+
+The saved constraints are the constraints branch as the summary saw it, not
constraints
+generated from the source SHA, so dependency bounds that changed between the
two can appear
+as conflicts or as stale constraints. The report summary shows both commits.
PyPI release
+metadata is fetched when the report executes, so the available upgrade targets
can change
+between the summary and the explanations.
+
+The workflow has read-only repository permissions. It runs on `workflow_run`,
in the context
+of the default branch, so it could write that branch's GitHub Actions cache:
what keeps the
+cache untouched is that no step saves one. Do not add caching (for example
`actions/cache` or
+the cache option of a `setup-*` action) to this workflow. Reports use the
saved image artifact.
+
+The report runs at most three jobs concurrently. Each job has a 150-minute
limit and saves
+its output, including partial output on failure. A failed or timed-out report
is incomplete;
+it does not invalidate the canary's test result. Find diagnostics in the
**Dependency
+upgrade report** workflow, whose summary links to the source canary. To repeat
diagnostics,
+use **Re-run jobs** on that report before its two-day input retention expires
(or run
+`gh run rerun REPORT_RUN_ID`). Reruns keep the original source canary ID and
attempt. A rerun
+fails when inputs were saved but none can be explained any more, rather than
using a newer
+image or constraints; a report whose canary saved no inputs posts the same
notice every time.
+
+`breeze release-management constraints-version-check --constraints-file PATH`
can also
+analyze a saved constraints file locally. Detailed explanations reuse one
baseline resolution
+and skip the pinned resolution when that baseline already selects the target
version.
+
### Spot interruptions of self-hosted runners
Runs of `apache/airflow` committers and the canary run on self-hosted
diff --git
a/dev/breeze/doc/images/output_release-management_constraints-version-check.svg
b/dev/breeze/doc/images/output_release-management_constraints-version-check.svg
index 1289420d214..2164202499c 100644
---
a/dev/breeze/doc/images/output_release-management_constraints-version-check.svg
+++
b/dev/breeze/doc/images/output_release-management_constraints-version-check.svg
@@ -1,4 +1,4 @@
-<svg class="rich-terminal" viewBox="0 0 1482 782.0"
xmlns="http://www.w3.org/2000/svg";>
+<svg class="rich-terminal" viewBox="0 0 1482 879.5999999999999"
xmlns="http://www.w3.org/2000/svg";>
<!-- Generated with Rich https://www.textualize.io -->
<style>
@@ -43,7 +43,7 @@
<defs>
<clipPath
id="breeze-release-management-constraints-version-check-clip-terminal">
- <rect x="0" y="0" width="1463.0" height="731.0" />
+ <rect x="0" y="0" width="1463.0" height="828.5999999999999" />
</clipPath>
<clipPath id="breeze-release-management-constraints-version-check-line-0">
<rect x="0" y="1.5" width="1464" height="24.65"/>
@@ -132,9 +132,21 @@
<clipPath id="breeze-release-management-constraints-version-check-line-28">
<rect x="0" y="684.7" width="1464" height="24.65"/>
</clipPath>
+<clipPath id="breeze-release-management-constraints-version-check-line-29">
+ <rect x="0" y="709.1" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-constraints-version-check-line-30">
+ <rect x="0" y="733.5" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-constraints-version-check-line-31">
+ <rect x="0" y="757.9" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-constraints-version-check-line-32">
+ <rect x="0" y="782.3" width="1464" height="24.65"/>
+ </clipPath>
</defs>
- <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1"
x="1" y="1" width="1480" height="780" rx="8"/><text
class="breeze-release-management-constraints-version-check-title"
fill="#c5c8c6" text-anchor="middle" x="740"
y="27">Command: release-management constraints-version-check</text>
+ <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1"
x="1" y="1" width="1480" height="877.6" rx="8"/><text
class="breeze-release-management-constraints-version-check-title"
fill="#c5c8c6" text-anchor="middle" x="740"
y="27">Command: release-management constraints-version-check</text>
<g transform="translate(26,22)">
<circle cx="0" cy="0" r="7" fill="#ff5f57"/>
<circle cx="22" cy="0" r="7" fill="#febc2e"/>
@@ -155,25 +167,29 @@
</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="215.2" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-8)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="215.2" textLength="317.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-8)">--airflow-constraints-mode</text><text
class="breeze-release-management-constraints-version-check-r1" x="414.8" y
[...]
</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="239.6" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-9)">│</text><text
class="breeze-release-management-constraints-version-check-r5" x="414.8"
y="239.6" textLength="353.8"
clip-path="url(#breeze-release-management-constraints-version-check-line-9)">constraints-source-providers]</text><text
class="breeze-release-management-constraints-version-check-r7" x="780. [...]
</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="264" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-10)">│</text><text
class="breeze-release-management-constraints-version-check-r7" x="414.8"
y="264" textLength="305"
clip-path="url(#breeze-release-management-constraints-version-check-line-10)">constraints-no-providers)</text><text
class="breeze-release-management-constraints-version-check-r5" x="1451.8" y="2
[...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="288.4" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-11)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="288.4" textLength="317.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-11)">--github-repository       </text><text
class="breeze-release-management-cons [...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="312.8" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-12)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="312.8" textLength="317.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-12)">--github-token            </text><text
class="breez [...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="337.2" textLength="1464"
clip-path="url(#breeze-release-management-constraints-version-check-line-13)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-constraints-version-check-r1" x="1464"
y="337.2" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-13)">
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="361.6" textLength="24.4"
clip-path="url(#breeze-release-management-constraints-version-check-line-14)">╭─</text><text
class="breeze-release-management-constraints-version-check-r5" x="24.4"
y="361.6" textLength="219.6"
clip-path="url(#breeze-release-management-constraints-version-check-line-14)"> Comparison mode. </text><text
class="breeze-release-management-constraints-version-check-r5" x [...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="386" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-15)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="386" textLength="366"
clip-path="url(#breeze-release-management-constraints-version-check-line-15)">--diff-mode                 
[...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="410.4" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-16)">│</text><text
class="breeze-release-management-constraints-version-check-r7" x="414.8"
y="410.4" textLength="390.4"
clip-path="url(#breeze-release-management-constraints-version-check-line-16)">(full|diff-all|diff-constraints)</text><text
class="breeze-release-management-constraints-version-check-r5" x= [...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="434.8" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-17)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="434.8" textLength="366"
clip-path="url(#breeze-release-management-constraints-version-check-line-17)">--package                 &#
[...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="459.2" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-18)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="459.2" textLength="158.6"
clip-path="url(#breeze-release-management-constraints-version-check-line-18)">--explain-why</text><text
class="breeze-release-management-constraints-version-check-r1" x="183"
y="459.2" text [...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="483.6" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-19)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="483.6" textLength="366"
clip-path="url(#breeze-release-management-constraints-version-check-line-19)">--cooldown-days               </text><
[...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="508" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-20)">│</text><text
class="breeze-release-management-constraints-version-check-r5" x="414.8"
y="508" textLength="24.4"
clip-path="url(#breeze-release-management-constraints-version-check-line-20)">4]</text><text
class="breeze-release-management-constraints-version-check-r7" x="451.4"
y="508" textLength="109.8" [...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="532.4" textLength="1464"
clip-path="url(#breeze-release-management-constraints-version-check-line-21)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-constraints-version-check-r1" x="1464"
y="532.4" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-21)">
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="556.8" textLength="24.4"
clip-path="url(#breeze-release-management-constraints-version-check-line-22)">╭─</text><text
class="breeze-release-management-constraints-version-check-r5" x="24.4"
y="556.8" textLength="195.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-22)"> Build options. </text><text
class="breeze-release-management-constraints-version-check-r5" x=" [...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="581.2" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-23)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="581.2" textLength="109.8"
clip-path="url(#breeze-release-management-constraints-version-check-line-23)">--builder</text><text
class="breeze-release-management-constraints-version-check-r1" x="158.6"
y="581.2" textLe [...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="605.6" textLength="1464"
clip-path="url(#breeze-release-management-constraints-version-check-line-24)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-constraints-version-check-r1" x="1464"
y="605.6" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-24)">
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="630" textLength="24.4"
clip-path="url(#breeze-release-management-constraints-version-check-line-25)">╭─</text><text
class="breeze-release-management-constraints-version-check-r5" x="24.4"
y="630" textLength="195.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-25)"> Common options </text><text
class="breeze-release-management-constraints-version-check-r5" x="219. [...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="654.4" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-26)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="654.4" textLength="109.8"
clip-path="url(#breeze-release-management-constraints-version-check-line-26)">--verbose</text><text
class="breeze-release-management-constraints-version-check-r6" x="158.6"
y="654.4" textLe [...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="678.8" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-27)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="678.8" textLength="109.8"
clip-path="url(#breeze-release-management-constraints-version-check-line-27)">--dry-run</text><text
class="breeze-release-management-constraints-version-check-r6" x="158.6"
y="678.8" textLe [...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="703.2" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-28)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="703.2" textLength="109.8"
clip-path="url(#breeze-release-management-constraints-version-check-line-28)">--help   </text><text
class="breeze-release-management-constraints-version-check-r6" x="158.6" y
[...]
-</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="727.6" textLength="1464"
clip-path="url(#breeze-release-management-constraints-version-check-line-29)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-constraints-version-check-r1" x="1464"
y="727.6" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-29)">
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="288.4" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-11)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="288.4" textLength="317.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-11)">--constraints-file        </text><text
class="breeze-release-management [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="312.8" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-12)">│</text><text
class="breeze-release-management-constraints-version-check-r1" x="414.8"
y="312.8" textLength="97.6"
clip-path="url(#breeze-release-management-constraints-version-check-line-12)">branch. </text><text
class="breeze-release-management-constraints-version-check-r7" x="512.4"
y="312.8" te [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="337.2" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-13)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="337.2" textLength="317.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-13)">--save-constraints-to     </text><text
class="breeze-release-management-constraints-ve [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="361.6" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-14)">│</text><text
class="breeze-release-management-constraints-version-check-r1" x="414.8"
y="361.6" textLength="817.4"
clip-path="url(#breeze-release-management-constraints-version-check-line-14)">to save them is reported as a warning and does not fail
[...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="386" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-15)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="386" textLength="317.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-15)">--github-repository       </text><text
class="breeze-release-management-constrai [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="410.4" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-16)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="410.4" textLength="317.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-16)">--github-token            </text><text
class="breez [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="434.8" textLength="1464"
clip-path="url(#breeze-release-management-constraints-version-check-line-17)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-constraints-version-check-r1" x="1464"
y="434.8" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-17)">
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="459.2" textLength="24.4"
clip-path="url(#breeze-release-management-constraints-version-check-line-18)">╭─</text><text
class="breeze-release-management-constraints-version-check-r5" x="24.4"
y="459.2" textLength="219.6"
clip-path="url(#breeze-release-management-constraints-version-check-line-18)"> Comparison mode. </text><text
class="breeze-release-management-constraints-version-check-r5" x [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="483.6" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-19)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="483.6" textLength="366"
clip-path="url(#breeze-release-management-constraints-version-check-line-19)">--diff-mode                 
[...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="508" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-20)">│</text><text
class="breeze-release-management-constraints-version-check-r7" x="414.8"
y="508" textLength="390.4"
clip-path="url(#breeze-release-management-constraints-version-check-line-20)">(full|diff-all|diff-constraints)</text><text
class="breeze-release-management-constraints-version-check-r5" x="145 [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="532.4" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-21)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="532.4" textLength="366"
clip-path="url(#breeze-release-management-constraints-version-check-line-21)">--package                 &#
[...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="556.8" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-22)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="556.8" textLength="158.6"
clip-path="url(#breeze-release-management-constraints-version-check-line-22)">--explain-why</text><text
class="breeze-release-management-constraints-version-check-r1" x="183"
y="556.8" text [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="581.2" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-23)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="581.2" textLength="366"
clip-path="url(#breeze-release-management-constraints-version-check-line-23)">--cooldown-days               </text><
[...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="605.6" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-24)">│</text><text
class="breeze-release-management-constraints-version-check-r5" x="414.8"
y="605.6" textLength="24.4"
clip-path="url(#breeze-release-management-constraints-version-check-line-24)">4]</text><text
class="breeze-release-management-constraints-version-check-r7" x="451.4"
y="605.6" textLength="1 [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="630" textLength="1464"
clip-path="url(#breeze-release-management-constraints-version-check-line-25)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-constraints-version-check-r1" x="1464"
y="630" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-25)">
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="654.4" textLength="24.4"
clip-path="url(#breeze-release-management-constraints-version-check-line-26)">╭─</text><text
class="breeze-release-management-constraints-version-check-r5" x="24.4"
y="654.4" textLength="195.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-26)"> Build options. </text><text
class="breeze-release-management-constraints-version-check-r5" x=" [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="678.8" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-27)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="678.8" textLength="109.8"
clip-path="url(#breeze-release-management-constraints-version-check-line-27)">--builder</text><text
class="breeze-release-management-constraints-version-check-r1" x="158.6"
y="678.8" textLe [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="703.2" textLength="1464"
clip-path="url(#breeze-release-management-constraints-version-check-line-28)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-constraints-version-check-r1" x="1464"
y="703.2" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-28)">
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="727.6" textLength="24.4"
clip-path="url(#breeze-release-management-constraints-version-check-line-29)">╭─</text><text
class="breeze-release-management-constraints-version-check-r5" x="24.4"
y="727.6" textLength="195.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-29)"> Common options </text><text
class="breeze-release-management-constraints-version-check-r5" x=" [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="752" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-30)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="752" textLength="109.8"
clip-path="url(#breeze-release-management-constraints-version-check-line-30)">--verbose</text><text
class="breeze-release-management-constraints-version-check-r6" x="158.6"
y="752" textLength=" [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="776.4" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-31)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="776.4" textLength="109.8"
clip-path="url(#breeze-release-management-constraints-version-check-line-31)">--dry-run</text><text
class="breeze-release-management-constraints-version-check-r6" x="158.6"
y="776.4" textLe [...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="800.8" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-32)">│</text><text
class="breeze-release-management-constraints-version-check-r4" x="24.4"
y="800.8" textLength="109.8"
clip-path="url(#breeze-release-management-constraints-version-check-line-32)">--help   </text><text
class="breeze-release-management-constraints-version-check-r6" x="158.6" y
[...]
+</text><text class="breeze-release-management-constraints-version-check-r5"
x="0" y="825.2" textLength="1464"
clip-path="url(#breeze-release-management-constraints-version-check-line-33)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-constraints-version-check-r1" x="1464"
y="825.2" textLength="12.2"
clip-path="url(#breeze-release-management-constraints-version-check-line-33)">
</text>
</g>
</g>
diff --git
a/dev/breeze/doc/images/output_release-management_constraints-version-check.txt
b/dev/breeze/doc/images/output_release-management_constraints-version-check.txt
index 7ed9535bd4c..78700f82fb0 100644
---
a/dev/breeze/doc/images/output_release-management_constraints-version-check.txt
+++
b/dev/breeze/doc/images/output_release-management_constraints-version-check.txt
@@ -1 +1 @@
-2c5c9bdbba4cde03848bc0c603c37d71
+c8bd7647f6101c627669eb67b8278432
diff --git
a/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
b/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
index 34138cea2f7..63edf1a7f4c 100644
--- a/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
+++ b/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
@@ -5020,6 +5020,17 @@ def publish_schemas_to_s3(
@option_builder
@option_python
@option_airflow_constraints_mode_ci
[email protected](
+ "--constraints-file",
+ type=click.Path(exists=True, dir_okay=False, path_type=Path),
+ help="Check this saved constraints file instead of downloading the current
constraints branch.",
+)
[email protected](
+ "--save-constraints-to",
+ type=click.Path(file_okay=False, path_type=Path),
+ help="Also save the downloaded constraints and their commit to this
directory. "
+ "A failure to save them is reported as a warning and does not fail the
check.",
+)
@click.option(
"--diff-mode",
type=click.Choice(["full", "diff-all", "diff-constraints"],
case_sensitive=False),
@@ -5051,6 +5062,8 @@ def publish_schemas_to_s3(
def version_check(
python: str,
airflow_constraints_mode: str,
+ constraints_file: Path | None,
+ save_constraints_to: Path | None,
diff_mode,
package: tuple[str],
explain_why: bool,
@@ -5059,6 +5072,11 @@ def version_check(
github_repository: str,
builder: str,
):
+ if constraints_file and save_constraints_to:
+ console_print(
+ "[error]--save-constraints-to saves downloaded constraints, not a
--constraints-file.[/]"
+ )
+ sys.exit(1)
perform_environment_checks()
fix_ownership_using_docker()
cleanup_python_generated_files()
@@ -5081,6 +5099,8 @@ def version_check(
github_token=github_token,
github_repository=github_repository,
cooldown_days=cooldown_days,
+ constraints_file=constraints_file,
+ save_constraints_to=save_constraints_to,
)
diff --git
a/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
b/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
index 94de79045ac..a8df438fd6f 100644
---
a/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
+++
b/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
@@ -609,6 +609,8 @@ RELEASE_MANAGEMENT_PARAMETERS: dict[str, list[dict[str, str
| list[str]]]] = {
"options": [
"--python",
"--airflow-constraints-mode",
+ "--constraints-file",
+ "--save-constraints-to",
"--github-repository",
"--github-token",
],
diff --git a/dev/breeze/src/airflow_breeze/utils/constraints_version_check.py
b/dev/breeze/src/airflow_breeze/utils/constraints_version_check.py
index 10c485fdd61..245d575740e 100755
--- a/dev/breeze/src/airflow_breeze/utils/constraints_version_check.py
+++ b/dev/breeze/src/airflow_breeze/utils/constraints_version_check.py
@@ -37,7 +37,7 @@ from airflow_breeze.global_constants import MOUNT_SELECTED
from airflow_breeze.params.shell_params import ShellParams
from airflow_breeze.utils.console import Output, console_print
from airflow_breeze.utils.docker_command_utils import execute_command_in_shell
-from airflow_breeze.utils.github import download_constraints_file
+from airflow_breeze.utils.github import download_constraints_file,
get_branch_commit
from airflow_breeze.utils.packages import load_pyproject_toml
from airflow_breeze.utils.parallel import get_temp_file_name
from airflow_breeze.utils.path_utils import AIRFLOW_ROOT_PATH
@@ -302,6 +302,20 @@ def get_first_newer_release_date_str(releases,
current_version):
return None
+def save_constraints(constraints: str, constraints_commit: str | None,
directory: Path) -> None:
+ """Keep the checked constraints for a later report. Failing to keep them
never fails the check."""
+ if not constraints_commit:
+ console_print("[warning]Constraints not saved: the constraints commit
could not be read.[/]")
+ return
+ try:
+ directory.mkdir(parents=True, exist_ok=True)
+ (directory / "constraints.txt").write_text(constraints)
+ # Written last, so that its presence means the copy is complete.
+ (directory /
"constraints-commit.txt").write_text(f"{constraints_commit}\n")
+ except OSError as e:
+ console_print(f"[warning]Constraints not saved to {directory}: {e}[/]")
+
+
def constraints_version_check(
python: str,
airflow_constraints_mode: str,
@@ -311,6 +325,8 @@ def constraints_version_check(
github_token: str | None = None,
github_repository: str | None = None,
cooldown_days: int = 4,
+ constraints_file: Path | None = None,
+ save_constraints_to: Path | None = None,
):
console_print(f"[bold cyan]Python version:[/] [white]{python}[/]")
console_print(f"[bold cyan]Constraints mode:[/]
[white]{airflow_constraints_mode}[/]")
@@ -320,16 +336,28 @@ def constraints_version_check(
f"[bold cyan]Cooldown period:[/] [white]{cooldown_days} days[/] "
f"[white]({exempt_count} distributions exempt via
exclude-newer-package)[/]\n"
)
- with tempfile.TemporaryDirectory() as temp_dir:
- constraints_file = Path(temp_dir) / "constraints.txt"
- download_constraints_file(
- constraints_reference=DEFAULT_AIRFLOW_CONSTRAINTS_BRANCH,
- python_version=python,
- airflow_constraints_mode=airflow_constraints_mode,
- github_token=github_token,
- output_file=constraints_file,
- )
+ if constraints_file is not None:
lines = constraints_file.read_text().splitlines()
+ else:
+ # A saved copy records its commit, so the branch is resolved first and
downloaded at that commit.
+ constraints_commit = (
+ get_branch_commit(DEFAULT_AIRFLOW_CONSTRAINTS_BRANCH, github_token)
+ if save_constraints_to
+ else None
+ )
+ with tempfile.TemporaryDirectory() as temp_dir:
+ downloaded_file = Path(temp_dir) / "constraints.txt"
+ download_constraints_file(
+ constraints_reference=constraints_commit or
DEFAULT_AIRFLOW_CONSTRAINTS_BRANCH,
+ python_version=python,
+ airflow_constraints_mode=airflow_constraints_mode,
+ github_token=github_token,
+ output_file=downloaded_file,
+ )
+ constraints = downloaded_file.read_text()
+ if save_constraints_to:
+ save_constraints(constraints, constraints_commit,
save_constraints_to)
+ lines = constraints.splitlines()
constraints_date = parse_constraints_generation_date(lines)
if constraints_date:
console_print(
@@ -804,12 +832,18 @@ def explain_package_upgrade(
explanation = (
f"[bold blue]\n--- Explaining for {pkg} (current: {pinned_version},
latest: {latest_version}) ---[/]"
)
- with preserve_files(AIRFLOW_ROOT_PATH / "pyproject.toml",
AIRFLOW_ROOT_PATH / "uv.lock"):
- from packaging.utils import canonicalize_name
+ # Breeze limits startup imports; packaging is needed only when explaining
a package.
+ from packaging.utils import canonicalize_name
+ canonical_pkg = str(canonicalize_name(pkg))
+ baseline_version = baseline_versions.get(canonical_pkg)
+ if baseline_version == latest_version:
+ return explanation + (
+ f"\n[bold green]Package {pkg} already resolves to {latest_version}
under "
+ f"--resolution highest. The constraints file appears to be
stale.[/]"
+ )
+ with preserve_files(AIRFLOW_ROOT_PATH / "pyproject.toml",
AIRFLOW_ROOT_PATH / "uv.lock"):
airflow_pyproject = AIRFLOW_ROOT_PATH / "pyproject.toml"
- canonical_pkg = str(canonicalize_name(pkg))
- baseline_version = baseline_versions.get(canonical_pkg)
update_pyproject_dependency(airflow_pyproject, pkg, latest_version,
python_version)
if get_verbose():
@@ -851,11 +885,6 @@ def explain_package_upgrade(
f"be read (empty freeze output), so the upgrade of {pkg} to
{latest_version} "
f"could not be classified.[/]"
)
- elif baseline_version == latest_version:
- explanation += (
- f"\n[bold green]Package {pkg} already resolves to
{latest_version} under "
- f"--resolution highest. The constraints file appears to be
stale.[/]"
- )
elif resolved_version != latest_version:
explanation += (
f"\n[bold yellow]uv sync succeeded but {pkg} still resolved to
"
diff --git a/dev/breeze/src/airflow_breeze/utils/github.py
b/dev/breeze/src/airflow_breeze/utils/github.py
index 57b675f50d3..f2e47125a5b 100644
--- a/dev/breeze/src/airflow_breeze/utils/github.py
+++ b/dev/breeze/src/airflow_breeze/utils/github.py
@@ -29,6 +29,7 @@ from typing import TYPE_CHECKING, Any, cast
from rich.markup import escape
+from airflow_breeze.global_constants import APACHE_AIRFLOW_GITHUB_REPOSITORY
from airflow_breeze.utils.confirm import Answer, user_confirm
from airflow_breeze.utils.console import console_print
from airflow_breeze.utils.path_utils import AIRFLOW_ROOT_PATH
@@ -180,10 +181,15 @@ def log_github_rate_limit_error(response: Response) ->
None:
def download_file_from_github(
- reference: str, path: str, output_file: Path, github_token: str | None =
None, timeout: int = 60
+ reference: str,
+ path: str,
+ output_file: Path,
+ github_token: str | None = None,
+ timeout: int = 60,
+ github_repository: str = APACHE_AIRFLOW_GITHUB_REPOSITORY,
) -> bool:
"""
- Downloads a file from the GitHub repository of Apache Airflow using the
GitHub API.
+ Downloads a file from a GitHub repository (Apache Airflow by default)
using the GitHub API.
In case of any error different from 404, it will exit the process with
error code 1.
@@ -192,11 +198,12 @@ def download_file_from_github(
:param output_file: Path where the file should be downloaded
:param github_token: GitHub token to use for authentication
:param timeout: timeout in seconds for the download request, default is 60
seconds
+ :param github_repository: repository to download from, default is
apache/airflow
:return: whether the file was successfully downloaded (False if the file
is missing)
"""
import requests
- url =
f"https://api.github.com/repos/apache/airflow/contents/{path}?ref={reference}";
+ url =
f"https://api.github.com/repos/{github_repository}/contents/{path}?ref={reference}";
console_print(f"[info]Downloading {url} to {output_file}")
if not get_dry_run():
headers = {"Accept": "application/vnd.github.v3.raw"}
@@ -299,6 +306,16 @@ def get_active_airflow_versions(
return airflow_versions, airflow_release_dates
+def get_constraints_github_repository() -> str:
+ """
+ Returns the repository keeping the constraints branches.
+
+ It is CONSTRAINTS_GITHUB_REPOSITORY when set, the variable the Dockerfiles
read for the same purpose,
+ and apache/airflow otherwise.
+ """
+ return os.environ.get("CONSTRAINTS_GITHUB_REPOSITORY") or
APACHE_AIRFLOW_GITHUB_REPOSITORY
+
+
def download_constraints_file(
constraints_reference: str,
python_version: str,
@@ -307,7 +324,7 @@ def download_constraints_file(
output_file: Path,
) -> bool:
"""
- Downloads constraints file from GitHub repository of Apache Airflow
+ Downloads constraints file from the constraints repository (see
get_constraints_github_repository)
:param constraints_reference: airflow version
:param python_version: python version
@@ -322,9 +339,42 @@ def download_constraints_file(
path=constraints_file_path,
github_token=github_token,
output_file=output_file,
+ github_repository=get_constraints_github_repository(),
)
+def get_branch_commit(branch: str, github_token: str | None = None, timeout:
int = 60) -> str | None:
+ """
+ Returns the commit a branch of the constraints repository (see
get_constraints_github_repository)
+ points to.
+
+ Unlike the downloads above, it never exits the process: any error is
reported as a warning.
+
+ :param branch: branch to resolve
+ :param github_token: GitHub token to use for authentication
+ :param timeout: timeout in seconds for the request, default is 60 seconds
+ :return: the commit SHA, or None if it could not be read
+ """
+ import requests
+
+ url =
f"https://api.github.com/repos/{get_constraints_github_repository()}/commits/{branch}"
+ headers = {"Accept": "application/vnd.github.sha"}
+ if github_token:
+ headers["Authorization"] = f"Bearer {github_token}"
+ headers["X-GitHub-Api-Version"] = "2022-11-28"
+ try:
+ response = requests.get(url, headers=headers, timeout=timeout)
+ except requests.RequestException as e:
+ console_print(f"[warning]Could not read the commit of {branch}: {e}")
+ return None
+ commit = response.text.strip()
+ if response.status_code != 200 or not re.fullmatch(r"[0-9a-f]{40}",
commit):
+ log_github_rate_limit_error(response)
+ console_print(f"[warning]Could not read the commit of {branch}. Status
code {response.status_code}")
+ return None
+ return commit
+
+
def get_tag_date(tag: str) -> str | None:
"""
Returns UTC timestamp of the tag in the repo in iso time format 8601
diff --git a/dev/breeze/tests/test_constraints_version_check.py
b/dev/breeze/tests/test_constraints_version_check.py
index 85856eb01fa..909fddfa0e3 100644
--- a/dev/breeze/tests/test_constraints_version_check.py
+++ b/dev/breeze/tests/test_constraints_version_check.py
@@ -24,7 +24,9 @@ from unittest import mock
import pytest
+from airflow_breeze.branch_defaults import DEFAULT_AIRFLOW_CONSTRAINTS_BRANCH
from airflow_breeze.utils.constraints_version_check import (
+ constraints_version_check,
explain_package_upgrade,
get_release_cutoff,
get_table_format,
@@ -39,6 +41,144 @@ OLD_UPLOAD_TIME = "2020-01-01T00:00:00.000000Z"
FRESH_UPLOAD_TIME = "2099-01-01T00:00:00.000000Z"
[email protected](f"{MODULE}.download_constraints_file", autospec=True)
[email protected](f"{MODULE}.load_cooldown_overrides", autospec=True,
return_value={})
+def test_saved_constraints_are_used_without_downloading_current_branch(
+ mock_overrides, mock_download, tmp_path, pypi
+):
+ constraints_file = tmp_path / "constraints.txt"
+ constraints_file.write_text("pkg-a==1.0.0\n")
+ constraints_version_check(
+ python="3.14",
+ airflow_constraints_mode="constraints-source-providers",
+ diff_mode="full",
+ constraints_file=constraints_file,
+ )
+ mock_download.assert_not_called()
+
+
[email protected](f"{MODULE}.download_constraints_file", autospec=True)
[email protected](f"{MODULE}.load_cooldown_overrides", autospec=True,
return_value={})
+def test_missing_saved_constraints_do_not_fall_back_to_current_branch(
+ mock_overrides, mock_download, tmp_path
+):
+ with pytest.raises(FileNotFoundError):
+ constraints_version_check(
+ python="3.14",
+ airflow_constraints_mode="constraints",
+ diff_mode="full",
+ constraints_file=tmp_path / "missing.txt",
+ )
+ mock_download.assert_not_called()
+
+
+def _download(*, output_file, **_):
+ output_file.write_text("pkg-a==1.0.0\n")
+ return True
+
+
[email protected](f"{MODULE}.download_constraints_file", autospec=True,
side_effect=_download)
[email protected](f"{MODULE}.get_branch_commit", autospec=True, return_value="c" *
40)
[email protected](f"{MODULE}.load_cooldown_overrides", autospec=True,
return_value={})
+def
test_downloaded_constraints_are_saved_with_the_commit_they_were_downloaded_at(
+ mock_overrides, mock_commit, mock_download, tmp_path, pypi
+):
+ constraints_version_check(
+ python="3.14",
+ airflow_constraints_mode="constraints",
+ diff_mode="full",
+ save_constraints_to=tmp_path / "saved",
+ )
+ assert mock_download.call_args.kwargs["constraints_reference"] == "c" * 40
+ assert (tmp_path / "saved" / "constraints.txt").read_text() ==
"pkg-a==1.0.0\n"
+ assert (tmp_path / "saved" / "constraints-commit.txt").read_text() == "c"
* 40 + "\n"
+
+
[email protected]("requests.get")
[email protected](f"{MODULE}.load_cooldown_overrides", autospec=True,
return_value={})
+def
test_saved_constraints_and_their_commit_come_from_constraints_github_repository(
+ mock_overrides, mock_get, monkeypatch, tmp_path, pypi
+):
+ monkeypatch.setenv("CONSTRAINTS_GITHUB_REPOSITORY", "my-org/airflow")
+ commit = "c" * 40
+ mock_get.side_effect = lambda url, **_: (
+ mock.Mock(status_code=200, text=commit)
+ if "/commits/" in url
+ else mock.Mock(status_code=200, content=b"pkg-a==1.0.0\n")
+ )
+ constraints_version_check(
+ python="3.14",
+ airflow_constraints_mode="constraints",
+ diff_mode="full",
+ save_constraints_to=tmp_path / "saved",
+ )
+ assert [call.args[0] for call in mock_get.call_args_list] == [
+
f"https://api.github.com/repos/my-org/airflow/commits/{DEFAULT_AIRFLOW_CONSTRAINTS_BRANCH}";,
+
f"https://api.github.com/repos/my-org/airflow/contents/constraints-3.14.txt?ref={commit}";,
+ ]
+ assert (tmp_path / "saved" / "constraints-commit.txt").read_text() ==
commit + "\n"
+
+
[email protected](f"{MODULE}.download_constraints_file", autospec=True,
side_effect=_download)
[email protected](f"{MODULE}.get_branch_commit", autospec=True, return_value=None)
[email protected](f"{MODULE}.load_cooldown_overrides", autospec=True,
return_value={})
+def test_constraints_are_still_checked_when_their_commit_cannot_be_read(
+ mock_overrides, mock_commit, mock_download, tmp_path, pypi
+):
+ constraints_version_check(
+ python="3.14",
+ airflow_constraints_mode="constraints",
+ diff_mode="full",
+ save_constraints_to=tmp_path / "saved",
+ )
+ assert mock_download.call_args.kwargs["constraints_reference"] ==
DEFAULT_AIRFLOW_CONSTRAINTS_BRANCH
+ assert not (tmp_path / "saved").exists()
+
+
[email protected](f"{MODULE}.download_constraints_file", autospec=True,
side_effect=_download)
[email protected](f"{MODULE}.get_branch_commit", autospec=True, return_value="c" *
40)
[email protected](f"{MODULE}.load_cooldown_overrides", autospec=True,
return_value={})
+def test_constraints_are_still_checked_when_they_cannot_be_saved(
+ mock_overrides, mock_commit, mock_download, tmp_path, pypi
+):
+ (tmp_path / "file").write_text("")
+ constraints_version_check(
+ python="3.14",
+ airflow_constraints_mode="constraints",
+ diff_mode="full",
+ save_constraints_to=tmp_path / "file" / "saved",
+ )
+ assert mock_download.call_count == 1
+
+
[email protected](f"{MODULE}.download_constraints_file", autospec=True,
side_effect=_download)
[email protected](f"{MODULE}.get_branch_commit", autospec=True)
[email protected](f"{MODULE}.load_cooldown_overrides", autospec=True,
return_value={})
+def test_constraints_branch_is_not_resolved_unless_saving(mock_overrides,
mock_commit, mock_download, pypi):
+ constraints_version_check(python="3.14",
airflow_constraints_mode="constraints", diff_mode="full")
+ mock_commit.assert_not_called()
+ assert mock_download.call_args.kwargs["constraints_reference"] ==
DEFAULT_AIRFLOW_CONSTRAINTS_BRANCH
+
+
[email protected](f"{MODULE}.update_pyproject_dependency", autospec=True)
[email protected](f"{MODULE}.sync_and_freeze", autospec=True)
[email protected]("package", ["pkg-a", "Pkg_A", "pkg.a"])
+def test_latest_baseline_version_needs_no_pinned_resolution(mock_sync,
mock_update, package):
+ explanation = explain_package_upgrade(
+ pkg=package,
+ pinned_version="1.0.0",
+ latest_version="2.0.0",
+ python_version="3.14",
+ airflow_constraints_mode="constraints",
+ github_repository="apache/airflow",
+ baseline_text="successful baseline",
+ baseline_versions={"pkg-a": "2.0.0"},
+ )
+ mock_sync.assert_not_called()
+ mock_update.assert_not_called()
+ assert "already resolves to 2.0.0" in explanation
+
+
def _pypi_payload(latest: str, *versions: str, fresh: str | None = None) ->
bytes:
releases = {
v: [{"upload_time_iso_8601": FRESH_UPLOAD_TIME if v == fresh else
OLD_UPLOAD_TIME, "yanked": False}]
diff --git a/dev/breeze/tests/test_dependency_upgrade_report.py
b/dev/breeze/tests/test_dependency_upgrade_report.py
new file mode 100644
index 00000000000..489d88d71c5
--- /dev/null
+++ b/dev/breeze/tests/test_dependency_upgrade_report.py
@@ -0,0 +1,396 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from __future__ import annotations
+
+import json
+import os
+import re
+import shutil
+import subprocess
+
+import pytest
+import yaml
+
+from airflow_breeze.global_constants import RUNS_ON_AMD_RUNNER_TEMPLATE
+from airflow_breeze.utils.path_utils import AIRFLOW_ROOT_PATH
+
+NODE_HARNESS = """
+import { readFileSync } from 'node:fs';
+const input = JSON.parse(readFileSync(0, 'utf8'));
+process.env.SOURCE_RUN_ID = input.id;
+process.env.SOURCE_RUN_ATTEMPT = input.sourceAttempt;
+process.env.GITHUB_RUN_ATTEMPT = String(input.attempt);
+process.env.GITHUB_RUN_ID = '456';
+const result = { outputs: {}, requests: [], notices: [], warnings: [],
summary: '' };
+const summary = {
+ addHeading() { return this; }, addLink() { return this; },
+ addRaw(text) { result.summary += text; return this; }, async write() {}
+};
+const github = {
+ rest: { actions: {
+ async getWorkflowRunAttempt(args) { result.requests.push(args); return {
data: input.run }; },
+ listWorkflowRunArtifacts() {}
+ } },
+ async paginate(method, args) { result.requests.push(args); return
input.artifacts; }
+};
+const core = {
+ setOutput(key, value) { result.outputs[key] = value; }, summary,
+ notice(message) { result.notices.push(message); },
+ warning(message) { result.warnings.push(message); }
+};
+const context = { repo: { owner: 'apache', repo: 'airflow' } };
+const AsyncFunction = Object.getPrototypeOf(async function () {}).constructor;
+try {
+ await new AsyncFunction('github', 'context', 'core', input.script)(github,
context, core);
+} catch (error) {
+ result.error = error.message;
+}
+console.log(JSON.stringify(result));
+"""
+
+NO_INPUTS_NOTICE = "No saved report inputs: this run did not request
independent diagnostics"
+
+requires_node = pytest.mark.skipif(shutil.which("node") is None, reason="The
plan script runs in node")
+
+
[email protected]
+def source_run():
+ return {
+ "head_repository": {"full_name": "apache/airflow"},
+ "head_branch": "main",
+ "path": ".github/workflows/ci-amd.yml",
+ "event": "schedule",
+ "status": "completed",
+ "head_sha": "a" * 40,
+ "html_url": "https://github.com/apache/airflow/actions/runs/123";,
+ }
+
+
[email protected]
+def source_artifacts():
+ return [
+ {
+ "id": 1,
+ "name":
"dependency-report-inputs-3.14-constraints-source-providers-true-1",
+ "expired": False,
+ },
+ {"id": 2, "name": "ci-image-save-v3-linux_amd64-3.14-main", "expired":
False},
+ ]
+
+
+def load_workflow(path):
+ return yaml.safe_load((AIRFLOW_ROOT_PATH / ".github" / path).read_text())
+
+
+def summary_steps():
+ steps =
load_workflow("workflows/finalize-tests.yml")["jobs"]["dependency-upgrade-summary"]["steps"]
+ return {step["name"]: step for step in steps}
+
+
+def run_plan(source_run, source_artifacts, *, attempt=1, source_run_id="123",
source_attempt="1"):
+ workflow = load_workflow("workflows/dependency-upgrade-report.yml")
+ script = workflow["jobs"]["plan"]["steps"][0]["with"]["script"]
+ result = subprocess.run(
+ ["node", "--input-type=module", "--eval", NODE_HARNESS],
+ input=json.dumps(
+ {
+ "script": script,
+ "id": source_run_id,
+ "sourceAttempt": source_attempt,
+ "attempt": attempt,
+ "run": source_run,
+ "artifacts": source_artifacts,
+ }
+ ),
+ capture_output=True,
+ text=True,
+ check=True,
+ timeout=20,
+ )
+ return json.loads(result.stdout)
+
+
+def included_inputs(result):
+ return [entry["inputs-id"] for entry in
json.loads(result["outputs"]["matrix"])["include"]]
+
+
+@requires_node
[email protected]("attempt", [1, 2])
+def test_plan_keeps_original_inputs_when_the_report_is_rerun(source_run,
source_artifacts, attempt):
+ result = run_plan(source_run, source_artifacts, attempt=attempt)
+ outputs = result["outputs"]
+ assert outputs["source-run-id"] == "123"
+ assert outputs["source-sha"] == source_run["head_sha"]
+ assert outputs["has-reports"] == "true"
+ assert json.loads(outputs["matrix"])["include"] == [
+ {
+ "python": "3.14",
+ "mode": "constraints-source-providers",
+ "use-uv": "true",
+ "inputs-id": 1,
+ "image-id": 2,
+ "image-source": "artifact",
+ "runner": "ubuntu-22.04",
+ }
+ ]
+ assert all(request["run_id"] == 123 for request in result["requests"])
+ assert result["requests"][0]["attempt_number"] == 1
+
+
+@requires_node
[email protected]("attempt", [1, 2, 3])
+def
test_plan_posts_the_same_notice_on_every_attempt_when_no_inputs_were_saved(source_run,
attempt):
+ result = run_plan(source_run, [], attempt=attempt)
+ assert "error" not in result
+ assert result["outputs"]["has-reports"] == "false"
+ assert result["notices"] == [NO_INPUTS_NOTICE]
+
+
+@requires_node
[email protected]("attempt", [2, 3])
+def test_plan_fails_a_rerun_when_original_inputs_are_gone(source_run,
source_artifacts, attempt):
+ result = run_plan(source_run, source_artifacts[:1], attempt=attempt)
+ assert result["error"] == "Saved report inputs are missing or expired"
+
+
[email protected](
+ ("field", "value"),
+ [
+ ("head_repository", {"full_name": "fork/airflow"}),
+ ("head_branch", "feature"),
+ ("event", "pull_request"),
+ ("path", ".github/workflows/other.yml"),
+ ("status", "in_progress"),
+ ],
+)
+@requires_node
+def test_plan_rejects_sources_outside_completed_main_canaries(source_run,
source_artifacts, field, value):
+ source_run[field] = value
+ result = run_plan(source_run, source_artifacts)
+ assert result["error"] == "Only completed main AMD canary runs are
supported"
+
+
+@requires_node
[email protected](
+ ("missing", "reason"),
+ [
+ ("image", "CI image missing"),
+ ("expired-image", "CI image expired"),
+ ("expired-input", "inputs expired"),
+ ],
+)
+def test_plan_does_not_substitute_newer_artifacts(source_run,
source_artifacts, missing, reason):
+ if missing == "image":
+ source_artifacts.pop(1)
+ elif missing == "expired-image":
+ source_artifacts[1]["expired"] = True
+ else:
+ source_artifacts[0]["expired"] = True
+ source_artifacts += [
+ {"id": 3, "name": "dependency-report-inputs-3.13-constraints-true-1",
"expired": False},
+ {"id": 4, "name": "ci-image-save-v3-linux_amd64-3.13-main", "expired":
False},
+ ]
+ result = run_plan(source_run, source_artifacts)
+ skipped = f"3.14:constraints-source-providers ({reason})"
+ assert result["warnings"] == [
+ f"Skipping 3.14:constraints-source-providers: original inputs or CI
image unavailable ({reason})"
+ ]
+ assert f"Skipped: {skipped}" in result["summary"]
+ assert included_inputs(result) == [3]
+
+
+@requires_node
[email protected](("source_attempt", "inputs_ids"), [("1", [1]), ("2",
[3])])
+def
test_plan_explains_inputs_only_for_the_first_canary_attempt_that_saved_them(
+ source_run, source_artifacts, source_attempt, inputs_ids
+):
+ source_artifacts += [
+ {
+ "id": 5,
+ "name":
"dependency-report-inputs-3.14-constraints-source-providers-true-2",
+ "expired": False,
+ },
+ {"id": 3, "name": "dependency-report-inputs-3.13-constraints-true-2",
"expired": False},
+ {"id": 4, "name": "ci-image-save-v3-linux_amd64-3.13-main", "expired":
False},
+ ]
+ result = run_plan(source_run, source_artifacts,
source_attempt=source_attempt)
+ assert result["requests"][0]["attempt_number"] == int(source_attempt)
+ assert included_inputs(result) == inputs_ids
+ if source_attempt == "2":
+ assert "Saved by an earlier attempt:
3.14:constraints-source-providers" in result["summary"]
+
+
+@requires_node
+def
test_plan_does_not_queue_a_second_report_from_inputs_already_explained(source_run,
source_artifacts):
+ source_artifacts.append(
+ {
+ "id": 5,
+ "name":
"dependency-report-inputs-3.14-constraints-source-providers-true-2",
+ "expired": False,
+ }
+ )
+ result = run_plan(source_run, source_artifacts, source_attempt="2")
+ assert result["outputs"]["has-reports"] == "false"
+ assert result["notices"] == ["The saved report inputs were already saved
by an earlier canary attempt"]
+
+
+def stash_artifact_name(key, ref):
+ """The artifact name apache/infrastructure-actions/stash/save gives a
stash (its shared/mung.py)."""
+ return "-".join(re.sub(r"[^_\-.\w]", "_", part) for part in (key, ref))
+
+
+def image_stash_key():
+ steps =
load_workflow("workflows/ci-image-build.yml")["jobs"]["build-ci-images"]["steps"]
+ keys = [
+ step["with"]["key"]
+ for step in steps
+ if step.get("uses", "").endswith("/stash-save")
+ and step["with"]["key"].startswith("ci-image-save-")
+ and "image-stash-ref" not in step["with"]["key"]
+ ]
+ assert keys == ["ci-image-save-v3-${{ inputs.platform }}-${{
env.PYTHON_MAJOR_MINOR_VERSION }}"]
+ key = keys[0].replace("${{ inputs.platform }}", "linux/amd64")
+ return key.replace("${{ env.PYTHON_MAJOR_MINOR_VERSION }}", "3.14")
+
+
+def runs_on_digest_suffix():
+ """The suffix .github/actions/stash-save adds to the key of the digest it
stashes on RunsOn."""
+ steps = load_workflow("actions/stash-save/action.yml")["runs"]["steps"]
+ digest = next(step for step in steps if step["name"] == "Save stash digest
to GitHub artifacts")
+ return digest["with"]["key"].removeprefix("${{ inputs.key }}")
+
+
+@requires_node
[email protected](
+ ("runs_on", "image_source", "runner"),
+ [
+ pytest.param(False, "artifact", "ubuntu-22.04",
id="github-hosted-canary"),
+ pytest.param(
+ True, "s3",
RUNS_ON_AMD_RUNNER_TEMPLATE.format(run_id="456").format(8, 32),
id="runs-on-canary"
+ ),
+ ],
+)
+def test_plan_finds_the_image_under_the_name_the_canary_stashes_it(
+ source_run, source_artifacts, runs_on, image_source, runner
+):
+ key = image_stash_key() + (runs_on_digest_suffix() if runs_on else "")
+ source_artifacts[1]["name"] = stash_artifact_name(key, "main")
+ [entry] = json.loads(run_plan(source_run,
source_artifacts)["outputs"]["matrix"])["include"]
+ assert entry["inputs-id"] == 1
+ assert entry["image-id"] == 2
+ assert entry["image-source"] == image_source
+ assert entry["runner"] == runner
+
+
+def test_explain_restores_a_runs_on_image_where_stash_save_put_it():
+ stash_save = (AIRFLOW_ROOT_PATH / ".github" / "actions" / "stash-save" /
"action.yml").read_text()
+ assert 'prefix="s3://${bucket}/cache/airflow-stash"' in stash_save
+ assert 'object="${prefix}/sha256/${digest}.tar"' in stash_save
+ steps =
load_workflow("workflows/dependency-upgrade-report.yml")["jobs"]["explain"]["steps"]
+ by_name = {step["name"]: step for step in steps}
+ restore = by_name["Restore the original CI image from the RunsOn cache"]
+ assert 'object="s3://${bucket}/cache/airflow-stash/sha256/${digest}.tar"'
in restore["run"]
+ assert "sha256sum --check" in restore["run"]
+ assert restore["if"] == "matrix.image-source == 's3'"
+ assert by_name["Download the digest of the original CI image"]["if"] ==
"matrix.image-source == 's3'"
+ assert by_name["Download the original CI image"]["if"] ==
"matrix.image-source == 'artifact'"
+ assert
load_workflow("workflows/dependency-upgrade-report.yml")["jobs"]["explain"]["runs-on"]
== (
+ "${{ matrix.runner }}"
+ )
+
+
+def test_each_canary_attempt_triggers_its_own_report():
+ workflow = load_workflow("workflows/dependency-upgrade-report.yml")
+ assert workflow["concurrency"]["group"].endswith("-${{
github.event.workflow_run.run_attempt }}")
+ assert workflow["concurrency"]["cancel-in-progress"] is False
+ plan_env = workflow["jobs"]["plan"]["steps"][0]["env"]
+ assert plan_env["SOURCE_RUN_ATTEMPT"] == "${{
github.event.workflow_run.run_attempt }}"
+ upload = summary_steps()["Upload dependency report inputs"]
+ assert upload["with"]["name"].endswith("-${{ github.run_attempt }}")
+
+
+def test_explain_prepares_the_disk_like_the_image_build_jobs():
+ build_steps =
load_workflow("workflows/ci-image-build.yml")["jobs"]["build-ci-images"]["steps"]
+ steps =
load_workflow("workflows/dependency-upgrade-report.yml")["jobs"]["explain"]["steps"]
+ names = [step["name"] for step in steps]
+ workflow_version = "Checkout the workflow version for its actions and
runner scripts"
+ disk_steps = ["Free up disk space", "Make /mnt writeable", "Move docker to
/mnt"]
+ assert names[1:6] == [workflow_version, *disk_steps, "Install Breeze"]
+ assert names.index("Install Breeze") < names.index("Download the original
CI image")
+ build = {step["name"]: step for step in build_steps}
+ assert steps[names.index(workflow_version)]["with"] ==
build[workflow_version]["with"]
+ for name in disk_steps:
+ assert steps[names.index(name)]["run"] == build[name]["run"]
+
+
[email protected](
+ ("independent_report", "report_args"),
+ [
+ ("true", ["--no-explain-why", "--save-constraints-to",
"files/dependency-report"]),
+ ("false", ["--explain-why"]),
+ ],
+)
+def test_canary_saves_report_inputs_through_breeze(tmp_path,
independent_report, report_args):
+ steps = summary_steps()
+ assert not any("gh api" in step.get("run", "") for step in steps.values())
+ upload = steps["Upload dependency report inputs"]
+ assert "hashFiles('files/dependency-report/constraints-commit.txt')" in
upload["if"]
+ assert upload["continue-on-error"] is True
+ deps = next(step for name, step in steps.items() if name.startswith("Deps:
"))
+ (tmp_path / "breeze").write_text('#!/usr/bin/env bash\nprintf "%s\\n"
"$@"\n')
+ (tmp_path / "breeze").chmod(0o755)
+ result = subprocess.run(
+ ["bash", "--noprofile", "--norc", "-eo", "pipefail", "-c",
deps["run"]],
+ env={
+ **os.environ,
+ "PATH": f"{tmp_path}{os.pathsep}{os.environ['PATH']}",
+ "INDEPENDENT_REPORT": independent_report,
+ "MATRIX_PYTHON_VERSION": "3.14",
+ "MATRIX_CONSTRAINTS_MODE": "constraints",
+ },
+ capture_output=True,
+ text=True,
+ check=True,
+ timeout=20,
+ )
+ assert result.stdout.splitlines() == [
+ "release-management",
+ "constraints-version-check",
+ "--python",
+ "3.14",
+ "--airflow-constraints-mode",
+ "constraints",
+ *report_args,
+ ]
+
+
+@requires_node
[email protected]("source_run_id", ["0", "abc", "123\ninjected",
"9007199254740992"])
+def test_plan_rejects_invalid_source_ids_before_calling_github(source_run,
source_artifacts, source_run_id):
+ result = run_plan(source_run, source_artifacts,
source_run_id=source_run_id)
+ assert result["error"] == "Invalid source run ID"
+ assert result["requests"] == []
+
+
+@requires_node
[email protected]("source_attempt", ["0", "abc", "1\ninjected",
"12345"])
+def test_plan_rejects_invalid_source_attempts_before_calling_github(
+ source_run, source_artifacts, source_attempt
+):
+ result = run_plan(source_run, source_artifacts,
source_attempt=source_attempt)
+ assert result["error"] == "Invalid source run attempt"
+ assert result["requests"] == []
diff --git a/dev/breeze/tests/test_github_utils.py
b/dev/breeze/tests/test_github_utils.py
index 3b2e2395f3e..c675d7a9f5a 100644
--- a/dev/breeze/tests/test_github_utils.py
+++ b/dev/breeze/tests/test_github_utils.py
@@ -22,8 +22,11 @@ from unittest import mock
import pytest
from airflow_breeze.utils.github import (
+ download_constraints_file,
+ download_file_from_github,
env_without_github_tokens,
format_github_token_scope_guidance,
+ get_branch_commit,
retrieve_github_token,
run_gh_command,
)
@@ -182,3 +185,81 @@ def
test_run_gh_command_skips_subprocess_in_dry_run(mock_run):
assert result.returncode == 0
mock_run.assert_not_called()
+
+
[email protected]("requests.get")
+def test_get_branch_commit_returns_the_commit_sha(mock_get, monkeypatch):
+ monkeypatch.delenv("CONSTRAINTS_GITHUB_REPOSITORY", raising=False)
+ mock_get.return_value = mock.Mock(status_code=200, text="c" * 40 + "\n")
+
+ assert get_branch_commit("constraints-main", github_token="token") == "c"
* 40
+ assert (
+ mock_get.call_args.args[0] ==
"https://api.github.com/repos/apache/airflow/commits/constraints-main";
+ )
+ assert mock_get.call_args.kwargs["headers"]["Authorization"] == "Bearer
token"
+
+
[email protected](
+ "response",
+ [
+ mock.Mock(status_code=403, text="", headers={},
**{"json.return_value": {"message": "rate limit"}}),
+ mock.Mock(status_code=200, text="<html>not a sha</html>", headers={}),
+ ],
+)
[email protected]("requests.get")
+def test_get_branch_commit_returns_none_instead_of_exiting(mock_get, response):
+ mock_get.return_value = response
+
+ assert get_branch_commit("constraints-main") is None
+
+
[email protected]("requests.get")
+def test_get_branch_commit_returns_none_when_github_is_unreachable(mock_get):
+ import requests
+
+ mock_get.side_effect = requests.ConnectionError("unreachable")
+
+ assert get_branch_commit("constraints-main") is None
+
+
[email protected](
+ ("constraints_github_repository", "expected_repository"),
+ [(None, "apache/airflow"), ("", "apache/airflow"), ("my-org/airflow",
"my-org/airflow")],
+)
[email protected]("requests.get")
+def test_constraints_are_read_from_constraints_github_repository(
+ mock_get, constraints_github_repository, expected_repository, monkeypatch,
tmp_path
+):
+ if constraints_github_repository is None:
+ monkeypatch.delenv("CONSTRAINTS_GITHUB_REPOSITORY", raising=False)
+ else:
+ monkeypatch.setenv("CONSTRAINTS_GITHUB_REPOSITORY",
constraints_github_repository)
+ mock_get.return_value = mock.Mock(status_code=200, text="c" * 40,
content=b"pkg-a==1.0.0\n")
+
+ assert get_branch_commit("constraints-main") == "c" * 40
+ assert download_constraints_file(
+ constraints_reference="c" * 40,
+ python_version="3.14",
+ github_token=None,
+ airflow_constraints_mode="constraints",
+ output_file=tmp_path / "constraints.txt",
+ )
+ assert [call.args[0] for call in mock_get.call_args_list] == [
+
f"https://api.github.com/repos/{expected_repository}/commits/constraints-main";,
+
f"https://api.github.com/repos/{expected_repository}/contents/constraints-3.14.txt?ref={'c'
* 40}",
+ ]
+
+
[email protected]("requests.get")
+def test_other_files_are_still_downloaded_from_apache_airflow(mock_get,
monkeypatch, tmp_path):
+ monkeypatch.setenv("CONSTRAINTS_GITHUB_REPOSITORY", "my-org/airflow")
+ mock_get.return_value = mock.Mock(status_code=200, content=b"{}")
+
+ assert download_file_from_github(
+ reference="3.1.0",
+ path="airflow-core/src/airflow/ui/package.json",
+ output_file=tmp_path / "package.json",
+ )
+ assert mock_get.call_args.args[0] == (
+
"https://api.github.com/repos/apache/airflow/contents/airflow-core/src/airflow/ui/package.json?ref=3.1.0";
+ )
