This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new c31070368d Split PROD image verification to a separate step in CI
(#33140)
c31070368d is described below
commit c31070368df5688fc44bcacf3ce5978b32dbca3d
Author: Jarek Potiuk <[email protected]>
AuthorDate: Sat Aug 5 18:03:39 2023 +0200
Split PROD image verification to a separate step in CI (#33140)
The PROD image verification happens after the images are pulled
in the "wait for PROD images" step. This verification is pretty
helpful in detecting cases where there are some "installed airflow"
problems (for example recently it helped to avoid a circular import
problem in #33081 as one of the tests failed when images were
verified. However PROD image wait might fail for other reasons and
sometimes might be "neglected" as temporary failure.
Separating verification will allow to clearly surface that the
problem is with verification, not pulling the images.
---
.github/workflows/ci.yml | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index eedb6a3665..889ee7292d 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -1446,7 +1446,7 @@ jobs:
wait-for-prod-images:
timeout-minutes: 80
- name: "Wait for PROD images"
+ name: "Wait for & verify PROD images"
runs-on: "${{needs.build-info.outputs.runs-on}}"
needs: [build-info, wait-for-ci-images, build-prod-images]
if: needs.build-info.outputs.image-build == 'true'
@@ -1469,8 +1469,15 @@ jobs:
# We wait for the images to be available either from
"build-images.yml' run as pull_request_target
# or from build-prod-images above.
# We are utilising single job to wait for all images because this job
merely waits
- # For the images to be available and test them.
- run: breeze prod-image pull --verify --wait-for-image --run-in-parallel
+ # For the images to be available.
+ run: breeze prod-image pull --wait-for-image --run-in-parallel
+ env:
+ PYTHON_VERSIONS: ${{
needs.build-info.outputs.python-versions-list-as-string }}
+ DEBUG_RESOURCES: ${{needs.build-info.outputs.debug-resources}}
+ - name: Verify PROD images ${{ env.PYTHON_VERSIONS }}:${{ env.IMAGE_TAG
}}
+ # We pull images again (which is a NOOP) but this time we also verify
the images
+ # Having it as a separate step allows us to see if the problem was
with waiting or verification
+ run: breeze prod-image pull --verify --run-in-parallel
env:
PYTHON_VERSIONS: ${{
needs.build-info.outputs.python-versions-list-as-string }}
DEBUG_RESOURCES: ${{needs.build-info.outputs.debug-resources}}
@@ -1478,6 +1485,7 @@ jobs:
run: breeze ci fix-ownership
if: always()
+
test-docker-compose-quick-start:
timeout-minutes: 60
name: "Test docker-compose quick start"
