bvolpato commented on PR #40856: URL: https://github.com/apache/beam/pull/40856#issuecomment-6513688752
Concrete duplication evidence from the [October 28 scheduled snapshot run](https://github.com/apache/beam/actions/runs/40607579381), on `master` at `aea45221b65a0c2e6fef2bf5c8f0f6f3b0753dd1`: | Job | Requested task | Base-image task executed | |---|---|---| | [Standalone Python 3.11](https://github.com/apache/beam/actions/runs/40607579381/job/122550811442) | `:sdks:python:container:py311:docker` | `:sdks:python:container:py311:docker` at 00:20:35 UTC | | [Distroless Python 3.11](https://github.com/apache/beam/actions/runs/40607579381/job/122550811342) | `:sdks:python:container:distroless:py311:docker` | `:sdks:python:container:py311:docker` at 00:20:30 UTC, followed by the distroless task at 01:32:19 UTC | Both logs show the base-image task executing, without `UP-TO-DATE` or `FROM-CACHE`. Both also show a completed manifest push to `gcr.io/apache-beam-testing/beam-sdk/beam_python3.11_sdk:aea45221b65a0c2e6fef2bf5c8f0f6f3b0753dd1`. The standalone job finishes that push at 01:32:18 UTC, and the distroless job at 01:32:15 UTC. They produce different manifest digests, confirming separate builds and publication to the same image tag. The dependency is explicit in [distroless/common.gradle](https://github.com/apache/beam/blob/1760453f929ef75748693eaa0a89d37cb94ca4d8/sdks/python/container/distroless/common.gradle#L52): ```groovy dockerPrepare.dependsOn ":sdks:python:container:py${pythonVersionSuffix}:docker" ``` The remaining distroless matrix jobs therefore still run the matching base-image tasks. The [workflow arguments](https://github.com/apache/beam/blob/1760453f929ef75748693eaa0a89d37cb94ca4d8/.github/workflows/beam_Publish_Beam_SDK_Snapshots.yml#L119-L128) retain the registry, tag list, `arm64,amd64` platforms, and `-Ppush-containers`. This PR removes only the four standalone Python base-image matrix entries. In that run, the standalone jobs took [77m50s for Python 3.11](https://github.com/apache/beam/actions/runs/40607579381/job/122550811442), [98m44s for 3.12](https://github.com/apache/beam/actions/runs/40607579381/job/122550811479), [78m43s for 3.13](https://github.com/apache/beam/actions/runs/40607579381/job/122550811395), and [78m06s for 3.14](https://github.com/apache/beam/actions/runs/40607579381/job/122550811420). That is 5h33m23s of aggregate runner-job time. This is the observed cost of those jobs in one run. A completed publication run with this change is still needed to measure the actual reduction. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
