The PR is merged! You will need to rebase and rebuild your images now to use the hardened ones. This **might** cause temporary slower builds - but since most of it is just pulling the hardened docker image, it is way less disruption than occasionally rebuilding Python from sources. I hope this helps us to speed up the CI and local dev environment.
J. On Tue, Oct 20, 2026 at 12:44 AM Jarek Potiuk <[email protected]> wrote: > OK. So I understand we can proceed with the hardening change, and > separately we can discuss what tags we push to our DockerHub. > > I would love to get an approval on that one: > https://github.com/apache/airflow/pull/73038. It should also speed up the > builds in anumber of cases, including v3-4-test builds and documentation > builds, because we will avoid edge cases where Python has to be rebuilt > from sources. > > J. > > > On Thu, Oct 15, 2026 at 9:38 PM Jarek Potiuk <[email protected]> wrote: > >> Question (for Lazy consensus): >> >> Does anyone oppose the hardened image as a base? If so >> https://github.com/apache/airflow/pull/73038 could be approved and >> used for beta testing - it will likely speed up the work that Ilya is >> doing on Python 3.15 support. >> >> Then we can separately discuss whether to publish the legacy image, or >> even only decide to publish it if there will be isues with hardened >> image. The change in #73038 already allows to choose whether to build >> "hardened" or "legacy" image. >> >> J. >> >> >> J >> >> On Tue, Oct 13, 2026 at 10:37 AM Jarek Potiuk <[email protected]> wrote: >> > >> > Also - the https://github.com/apache/airflow/pull/73038 is green and >> ready to review/merge. It keeps both the old way of building images and the >> new hardened images. We can still choose (that will be a follow-up) which >> images to release. I hope we can reach a consensus on whether to release >> the -legacy one (it could have another name as well), so - for now I think >> merging this one and getting consensus on hardened images would be useful, >> allowing for more voices and discussion afterward. >> > >> > J. >> > >> > On Tue, Oct 13, 2026 at 10:09 AM Jarek Potiuk <[email protected]> wrote: >> >> >> >> It's a vanity metrics of course but we had 14M downloads in September >> as reported by Dockerhub. Like for other things we do not have more >> detailed data unfortunately. >> >> >> >> J. >> >> >> >> On Mon, Oct 12, 2026, 23:58 Vikram Koka via dev < >> [email protected]> wrote: >> >>> >> >>> I agree that many users may be extending the image and would like to >> see >> >>> some data if possible before making this call. >> >>> >> >>> On Mon, Oct 12, 2026 at 3:56 PM Vikram Koka <[email protected]> >> wrote: >> >>> >> >>> > Hmm, what data do we have on the use of our production images? >> >>> > >> >>> > For example for 3.3 vs. other versions over the last year? >> >>> > >> >>> > On Mon, Oct 12, 2026 at 4:09 AM Jarek Potiuk <[email protected]> >> wrote: >> >>> > >> >>> >> > I’m -5 to publishing the `-legacy` tag though. If someone wants >> that: >> >>> >> write your own Dockerfile and build it yourself. Our prod image >> should >> >>> >> not >> >>> >> try to be everything to everyone. >> >>> >> >> >>> >> If we do not publish the legacy image, migration to 3.4 will be >> harder for >> >>> >> a number of current users. In the past, there were issues where >> people >> >>> >> used >> >>> >> the image we published. >> >>> >> Many users extend the image by adding their dependencies, which >> might >> >>> >> cause >> >>> >> issues. The same occurred when we migrated to Bookworm two years >> ago: >> >>> >> libraries and dependencies change. Tools work a bit differently. >> >>> >> >> >>> >> But yes - we might not publish the image. We could simply tell >> people, as >> >>> >> we have done in the past, that they can build their own image >> instead of >> >>> >> us >> >>> >> publishing it ourselves. I am totally open to that. >> >>> >> >> >>> >> J. >> >>> >> >> >>> >> On Mon, Oct 12, 2026 at 10:23 AM Ash Berlin-Taylor <[email protected] >> > >> >>> >> wrote: >> >>> >> >> >>> >> > I’m +1 to just using latest Debian stable at the start of every >> minor >> >>> >> > release (I.e. 3.4.x will always be based on Debian 13) >> >>> >> > >> >>> >> > I’m -5 to publishing the `-legacy` tag though. If someone wants >> that: >> >>> >> > write your own Dockerfile and build it yourself. Our prod image >> should >> >>> >> not >> >>> >> > try to be everything to everyone. >> >>> >> > >> >>> >> > -ash >> >>> >> > >> >>> >> > >> >>> >> > > On 10 Oct 2026, at 11:02, Jarek Potiuk <[email protected]> >> wrote: >> >>> >> > > >> >>> >> > > Hello everyone, >> >>> >> > > >> >>> >> > > Following the discussion [1], I'd like to call for lazy >> consensus on >> >>> >> how >> >>> >> > > we build the CI and PROD images starting with Airflow 3.4.0: >> >>> >> > > >> >>> >> > > * 3.4.0: the CI image and the default PROD images are based on >> Docker >> >>> >> > > Hardened Images (Python prebuilt by Docker) on Debian 13 >> "trixie". >> >>> >> > > The base images are mirrored to >> ghcr.io/apache/airflow/base/python, >> >>> >> > > so building our images needs no registry credentials. >> >>> >> > > >> >>> >> > > * 3.4.x: we still publish PROD images built the current way >> (Python >> >>> >> > > compiled from sources by us on Debian 12 "bookworm"), but >> only under >> >>> >> a >> >>> >> > > new "-legacy" tag suffix (e.g. >> >>> >> apache/airflow:3.4.0-python3.12-legacy), >> >>> >> > > with a deprecation warning printed by the entrypoint. >> >>> >> > > >> >>> >> > > * 3.5.0: we drop the legacy images and bookworm entirely. >> >>> >> > > >> >>> >> > > * Testing: legacy images are not built in PRs or canary >> builds. A >> >>> >> weekly >> >>> >> > > scheduled job on main builds them and runs the PROD image >> checks, and >> >>> >> > > they are fully built and tested on the v3-4-test branch and >> before >> >>> >> > > every 3.4.x release. >> >>> >> > > >> >>> >> > > The implementation is in >> https://github.com/apache/airflow/pull/73038 >> >>> >> . >> >>> >> > > Breeze can build either flavor ("--image-flavor >> hardened|legacy", >> >>> >> > > "--debian-version trixie|bookworm"); the CI image is always >> hardened >> >>> >> > > and trixie-based. >> >>> >> > > >> >>> >> > > Two follow-ups will occur after this one merged: >> >>> >> > > >> >>> >> > > * integrating -legacy in CI/release framework >> >>> >> > > * security: signing our images cryptographically >> >>> >> > > >> >>> >> > > The last point is the benefit of using hardened images: they >> are >> >>> >> already >> >>> >> > > cryptographically signed, our images can build on top of that >> and >> >>> >> > > will be signed as well. with provenance information >> >>> >> > > so that our users will know the images were built using our >> secured >> >>> >> > > environment. I will implement this change by adding >> >>> >> > > a secure environment for all parts of the release process. >> >>> >> > > >> >>> >> > > The lazy consensus will last 72 hours, until 2026-10-13 18:00 >> UTC. If >> >>> >> > > there are no objections by then, I will consider it accepted. >> >>> >> > > >> >>> >> > > [1] >> https://lists.apache.org/thread/y0mlxsx36kzc84tn31q1kyl4rdvgv1xx >> >>> >> > > >> >>> >> > > J. >> >>> >> > > >> >>> >> > > >> --------------------------------------------------------------------- >> >>> >> > > To unsubscribe, e-mail: [email protected] >> >>> >> > > For additional commands, e-mail: [email protected] >> >>> >> > > >> >>> >> > >> >>> >> > >> >>> >> > >> --------------------------------------------------------------------- >> >>> >> > To unsubscribe, e-mail: [email protected] >> >>> >> > For additional commands, e-mail: [email protected] >> >>> >> > >> >>> >> > >> >>> >> >> >>> > >> >
