villebro commented on code in PR #44250:
URL: https://github.com/apache/superset/pull/44250#discussion_r4053645615
##########
.github/workflows/docker.yml:
##########
@@ -117,6 +221,11 @@ jobs:
matrix:
build_preset: ${{fromJson(needs.setup_matrix.outputs.matrix_config)}}
fail-fast: false
+ # Apply concurrency after change detection. A docs-only push has no
+ # docker-build job and cannot replace a pending publisher.
+ concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}-${{ matrix.build_preset
}}
+ cancel-in-progress: false
Review Comment:
Good catch, and this one was a straight regression on my side. Scoping
concurrency to the `docker-build` job in 9165f06627 applied
`cancel-in-progress: false` to every event, but `github.ref` on a PR is
`refs/pull/N/merge`, so every new commit landed in the same group and queued
behind the abandoned merge ref for up to the 60 minute job timeout.
Serialization is only needed on the publishing path.
Fixed in cf214febd7: `cancel-in-progress` is now `${{ github.event_name ==
'pull_request' }}`. Publishing pushes still queue so a slower run cannot
overwrite mutable tags; superseded `pull_request` validation builds cancel as
they did before.
##########
scripts/docker-build-extra-flags.sh:
##########
@@ -31,18 +31,53 @@
# appending PY_VER here would override supersetbot's pin and silently make
# "py311"/"py312" build the exact same image as "lean". Every other preset
# gets the override so its build lands on the Dockerfile's own supported
-# Python version.
+# Python version. Those presets also point buildx at a matching cache tag so
+# local and CI builds can pull cache layers for the same base image they build.
#
-# Usage: docker-build-extra-flags.sh <build_preset> <image_tag>
+# Usage: docker-build-extra-flags.sh <build_preset> [image_tag] [release]
set -euo pipefail
-BUILD_PRESET="${1:?usage: docker-build-extra-flags.sh <build_preset>
<image_tag>}"
-IMAGE_TAG="${2:?usage: docker-build-extra-flags.sh <build_preset> <image_tag>}"
+BUILD_PRESET="${1:?usage: docker-build-extra-flags.sh <build_preset>
[image_tag]}"
+IMAGE_TAG="${2:-}"
+BUILD_MODE="${3:-ci}"
+DEFAULT_PY_VER="$(sed -n 's/^ARG PY_VER=//p' Dockerfile | head -n 1)"
+if [ -z "$DEFAULT_PY_VER" ]; then
+ echo "Could not determine the default PY_VER from Dockerfile" >&2
+ exit 1
+fi
+
+EXTRA_FLAGS=""
+if [ "$BUILD_MODE" = "ci" ]; then
+ EXTRA_FLAGS="--build-arg INCLUDE_CHROMIUM=false"
+elif [ "$BUILD_MODE" != "release" ]; then
+ echo "Unknown Docker build mode: $BUILD_MODE" >&2
+ exit 1
+fi
+if [ -n "$IMAGE_TAG" ]; then
+ EXTRA_FLAGS="${EXTRA_FLAGS:+$EXTRA_FLAGS }--tag $IMAGE_TAG"
+fi
+case "$BUILD_PRESET" in
+ py311)
+ CACHE_REF="apache/superset-cache:3.11-slim-bookworm"
+ ;;
+ py312)
+ CACHE_REF="apache/superset-cache:3.12-slim-bookworm"
+ ;;
+ *)
+ CACHE_REF="apache/superset-cache:${DEFAULT_PY_VER}"
Review Comment:
Agreed, this PR created the divergence: the helper now derives the exported
tag from the Dockerfile while the five consumers still spell it out, so they
only happen to agree today.
Fixed in cf214febd7 with an invariant test,
`tests/unit_tests/scripts/docker_cache_ref_test.py`. It reads `ARG PY_VER` from
`Dockerfile` and regex-scans the three Compose files and two frontend workflows
for `apache/superset-cache:<tag>`, asserting every match equals the Dockerfile
default (allowing the `-dev`/`-lean` suffixes the helper appends for matrix
isolation). Scanning by pattern rather than an expected-value list means a
sixth consumer added later is covered automatically.
`scripts/docker-build-extra-flags.sh` is deliberately excluded, since its
`py311`/`py312` refs pin different base images on purpose.
I verified the negative case by temporarily bumping `ARG PY_VER` in the
Dockerfile: all five parametrized cases fail with a message pointing at the
exporter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
