To be perfectly honest, if we—as a community - want to take the risk and someone is ready to monitor issues related to it, B is also an acceptable option - even though I agree with Przemek that A would be safer.
Good thing is that we have the option now. We can **always** do it - and even retro-actively regenerate and publish such images. And we can treat it as a "reality check"—that is, how many people will start "shouting at us" because we broke their deployment. As long as we vote on it as a community and collectively take responsibility to monitor it, respond to people's issues, and react when we see it causing problems for many of our users, we will accept the risk that they walk out from Airflow rather than reporting issues to us. (We might become victims of "survivorship bias" and simply not hear from those who decided not to upgrade because it caused them a problem.). This is the question of the collective risk we are willing to accept. We can **just** leave instructions in the code how to enable the legacy build, how to rebuild it - so that - no matter if I or anyone else will be available, when it turns on we **need** to bring the images quickly - one of the maintainers will take on the task of re-publishing those. I remember what happened when MySQL broke their certificates (twice), which caused all our images to stop working (including those that our users just extend from)—we had a small tsunami of issues raised by people, and I spent almost the whole weekend recovering it. I certainly do not want that to happen again. And this is really what we are talking about: rather than building a "fallback solution" now - and telling our users how to use it, we defer it until the moment it happens (hoping it won't). It's a bit similar to "curing the disease when it happens" versus "preventing the disease by taking some precautions that cost us little - but certain time". We are trading a "certain" small investment now.+ some maintenance overhead for a "maybe" reactive approach that might cost much more (in terms of stress, potentially lost users, and plain time spent on fixing it). I would argue that, finally, maintaining even complex "supplementary" code - such as Dockerfiles—is several orders of magnitude cheaper with AI than it used to be. If those who vote "yes" accept that they might spend some hours responding to similar issues and rebuilding/regenerating the images, I am fine with B. Though my usual approach (A) is to "anticipate and prevent issues" - because I know that reactive time in the future, even if not very probable, costs a lot of my personal time that could otherwise be used for example to meet with friends. If I know I will not be the one scrambling to get old images back because of a sudden issue, I am happy. This is what +1 / -1 here really means. I am happy to call for a vote if we think the discussion is finished. J. On Wed, Nov 4, 2026 at 7:21 PM Vikram Koka via dev <[email protected]> wrote: > This is interesting and the 14M from last month is significantly lower than > I expected. > > I agree that a fair number of users have platform teams for Airflow. From > what I seen, most of them build the docker images for their internal teams > so that they have precise controls including scanning the images for > security reasons. > > Especially with the introduction of multiple languages and the decoupled > Airflow core and Task SDKs in multiple languages, I would expect this > trend to accelerate. > > Based on these, I think we need to revise our approach to be more > lightweight. > Therefore, I lean towards option B. > > Vikram > > > On Wed, Nov 4, 2026 at 12:20 PM Przemysław Mirowski via dev < > [email protected]> wrote: > > > I would vote for option A. > > > > > Many users extend our base images with custom system and Python > > > dependencies. Upgrading the base OS changes library versions and tool > > > behavior, which may break some extended images. A -legacy tag allows > > > users to upgrade Airflow first and the OS later. However, maintaining > > > this requires building, testing, and releasing a second set of images > > > for every 3.4.x release, with the risk that users stall migration > > > until 3.5.0 (when bookworm support is dropped entirely). > > > > Users may stall with migration if we publish legacy images, but without > > them, we take away from users the possibility to do gradual changes on > > environments. First Airflow version to 3.4 and after that separate > > modification to DHI-based image. I assume that many users can have > > different issues, and it may be hard for us to predict all of them; > making > > that separation possible should: > > 1. make upgrading to 3.4.0 for users easier, which should mean quicker > > feedback (and adoption) > > 2. easier separation and debugging of potential problems in the > > environments thanks to separation of modifications introduced to the > > environments > > 3. users will have an easy path back in case of things not working after > > environment upgrade (no Airflow version revert needed) > > > > I think that if we do not publish legacy images, we will see slower > > adoption of the 3.4 version, which, I would say, is rather unwanted. > > > > The second part, which Jarek mentioned already, is stabilisation. This > can > > be on users' side and on ours too. We already encountered a couple of > > issues with new images, and I would assume that we will encounter more. > > Having both images published, we will have some safeguard if things go > bad > > with DHI. > > > > > - Vikram suggested checking usage data first. However, Docker Hub > > > only provides total pull counts (~14M in September), not > > > version-specific or usage breakdown, so community reports remain our > > > primary data source. > > > > We have to remember that these data will not be reliable and are missing > > context. Not reliable, because there are users who are using internal > > registries to which images from DockerHub are copied. We do not, and will > > not, have data on what is happening there. Missing context to what is > > mentioned already (no version specific, etc.) > > > > > DHI is relatively new, so in theory it'll become more stable as time > > goes on. > > > > I agree, but the question is how long it can potentially take. We don't > > know the answer to that question. It can be a month or a year. > > > > There is also one thing. Some users who are running Airflow may have a > > dedicated platform team for Airflow and some number of users. Having a > > possible split of business/platform change (Airflow version upgrade) and > > platform change (DHI) can make planning easier (not sure tho if we are > > looking at things like that 🤔, but I would consider organisational > > approach and people there too). > > > > I have nothing against dropping legacy with 3.5 release (at least now, it > > is hard to predict what situation will be in a couple months). > > > > Best, > > Przemek > > > > On 2026/11/02 12:13:02 Ash Berlin-Taylor via dev wrote: > > > Brain fart, sorry, I meant “which problems are these” but I have since > > seen your reply in the thread where I proposed simplification. > > > > > > > On 2 Nov 2026, at 11:45, Jarek Potiuk <[email protected]> wrote: > > > > > > > > The ones we build on? > > > > > > > > Docker Hardened Images (Python images). Generally wdely accepted as a > > way > > > > how to keep your images as secure as possible (they are patching > their > > > > images up to few times a day). > > > > > > > > They are freely available and free for commercial use - Apache 2.0 > > licensed > > > > (these are the community images we are using). > > > > > > > > They also have commercial FIPS compliant drop-in replacements, which > > people > > > > can use as base images when following our 'build your own image' > > recipe. > > > > > > > > This is a bonus because our users can easily achieve FIPS compliance > > and > > > > users often asked for it. So we do not have to provide it - they can > > pay > > > > Docker for access to those FIPS compliant replacement base images. > > > > > > > > More info: > > > > > > > > Hardened Images | Docker https://share.google/J7g1i0JXzdUGHkW6G > > > > > > > > J. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 2, 2026, 12:22 Ash Berlin-Taylor via dev < > > [email protected]> > > > > wrote: > > > > > > > >> What images are these? > > > >> > > > >>> On 1 Nov 2026, at 22:06, Jarek Potiuk <[email protected]> wrote: > > > >>> > > > >>> Actually - after seeing the kind of issues we faced even after my > > last > > > >>> email 2 days ago -> the issues (and complexity of solutions) might > be > > > >>> somewhat intimidating to our users > > > >>> > > > >>> I personally think we should provide a bridge for our users. > > > >>> > > > >>> While I **strongly** believe DHI approach is good and has strong > > security > > > >>> implications, some of my users for 3.4.* would prefer to use the > > > >> "original" > > > >>> setup. > > > >>> > > > >>> I am fine for 3.5 - to drop all kind of legacy images, but I think > > that > > > >>> might be to ""steep** change - and being able to just switch to > `FROM > > > >>> apache/aifrlow:legacy-3.11` is an easy way to address all kinds of > > > >>> incompatibilities. > > > >>> > > > >>> If we got (and we did) 3 library compatibility issues that > required a > > > >>> change in how the image uses the base DHI image - our users will > have > > > >> even > > > >>> more of that - and they will reach out to us. > > > >>> > > > >>> Are we ready to handle our user's problems? > > > >>> > > > >>> For me - the easiest approach would be: > > > >>> > > > >>> * here are some general ways and you might follow it > > > >>> * but if you have no time/energy to follow it - the legacy image > will > > > >> keep > > > >>> you running. > > > >>> > > > >>> I would always ask anyone who says "drop it": Are you ready to > solve > > > >>> problems of our users if they see problems? > > > >>> > > > >>> My answer is: Yes - try to fix it following our vague instructions, > > if > > > >> you > > > >>> fail, use the legacy image for an easy fallback. > > > >>> > > > >>> What is the answer if we decide not to publish legacy image ? Ash, > > Jed? > > > >>> what would you answer to the users? (and I am not precluding any > > answer - > > > >>> we just have to remember that this should be the answer PMC is > firmly > > > >>> behind, no matter who is answering it) > > > >>> > > > >>> On Fri, Oct 30, 2026 at 7:02 PM Jed Cunningham < > > [email protected] > > > >>> > > > >>> wrote: > > > >>> > > > >>>> I still lean toward B, myself. My take is that leaving C as an > > option > > > >>>> injects some uncertainty for both sides, and offering legacy > flavors > > > >> just > > > >>>> kicks the can down the road. > > > >>>> > > > >>>> DHI is relatively new, so in theory it'll become more stable as > time > > > >> goes > > > >>>> on. > > > >>>> > > > >> > > > >> > > > >> > --------------------------------------------------------------------- > > > >> To unsubscribe, e-mail: [email protected] > > > >> For additional commands, e-mail: [email protected] > > > >> > > > >> > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > >
