FYI - we just merged Aritra's change to build Python in CI image from sources https://github.com/apache/airflow/pull/52265 -> we will run it for a while in CI, i also want to test some self-upgrade scenarios.
Then we will look at applying it to the PROD image if we find no surprises. It looks really good and we might have slightly more secure images produced (i.e. getting read of some libraries CVEs faster than the "official" images. Good job Aritra! J. On Thu, Jun 26, 2025 at 9:16 PM Jarek Potiuk <ja...@potiuk.com> wrote: > It looks like building from sources on our CI using base debian image > (with enabled optimizations) "just works" and takes 1m36s. -> and taking > into account that we already have pretty sophisticated CI-level remote > caching that will allow to rebuild it only when either base image is > released or python is released - building sounds like a very good idea. > > J. > > On Thu, Jun 26, 2025 at 7:02 PM Damian Shaw <ds...@striketechnologies.com> > wrote: > >> Going from source is certainly going to give the most control, I >> personally choose python-build-standalone for my own images rather than >> building from source to outsource all the build choices, such as what >> optimization flags to enable, what compiler tool chains to use, etc. >> >> Damian >> >> >> >> -----Original Message----- >> From: Jarek Potiuk <ja...@potiuk.com> >> Sent: Thursday, June 26, 2025 10:15 AM >> To: dev@airflow.apache.org >> Subject: Re: [DISCUSS] Switching base Python container images to >> python-standalone ? >> >> Thanks Damian - very insightful. And yes - I did not really want to >> "diminish" the value of community images and work of the maintainers, it >> was really more on the "we base our image security on the assumptions that >> it's coming from "official" sources and surely there are some guardrails" - >> which turned out to be not really true. >> >> But.... We might not have to even use python standalone. >> >> Aritra just tested installation of Python straight from sources -> >> https://github.com/apache/airflow/pull/52265 and I was actually pretty >> surprised how non-problematic and fast it was - and it seems it passes all >> our CI tests. We just need to add a little pre-commit magic to get >> notified when we should update patchlevel version of Python when a new one >> is released, and we should be able to try it out in CI image - once it gets >> battle-tested with CI/breeze etc. we can transfer this to PROD image as >> well. We already have an idea how to do it - our PROD images are optimized >> for size and do not contain "build essentials" - but I think we should be >> able to build Python in the "build" segment and simply copy resulting >> binaries to the "main" segment - since in both cases we use the same base >> image, such 1-1 copy should **just work** - we already do the same with >> installed python packages - we install them (including building when >> needed) in build segment and we copy-over the installed .venv to the main >> segment. >> >> So ... we might even not need a discussion - installing from Python >> sources is THE BEST >> >> J. >> >> >> On Wed, Jun 25, 2025 at 6:25 PM Damian Shaw <ds...@striketechnologies.com >> > >> wrote: >> >> > First, I would like to thank the community members who have been >> > maintaining the Python docker images, it's one of those thankless >> > opensource infrastructure volunteer roles that they've been doing for >> > a long time. Unfortunately Docker assigns the title "Official Image" >> > for various community run images, which creates a misconception on the >> > guarantees being provided, and if I were a suspicious person I would >> > say Docker creates this misconception on purpose to both get free work >> > from community members and make Docker seem more supported by third >> > party organizations than it actually is. >> > >> > On the topic of python-build-standalone, I've been using it in >> > production for several months now and I'm fairly happy with it. >> > >> > However, one minor reproducibility issue I have when installing >> > python-build-standalone via uv, is that uv does not have an ability to >> > pin to a specific build between uv versions, as uv hard codes a >> > mapping of Python version and platform to a specific build and then >> > updates that mapping between releases. So, updating the version of uv >> > between runs, or having two users run different versions of uv to >> > initialize the environment can change the results. >> > >> > While normally such build changes are trivial, if you look at the uv >> > 0.7.8 and 0.7.9 changelogs you will see that sometimes they can have >> > significant >> > impact: https://github.com/astral-sh/uv/releases/tag/0.7.8. Also, this >> > email finally prompted make an issue on this topic: >> > https://github.com/astral-sh/uv/issues/14263. >> > >> > There are other ways to source python-build-standalone, such as >> > pbs-installer or writing your own script, but I've not yet spent any >> > time investigating them, so I can't comment on them. >> > >> > Damian >> > >> > -----Original Message----- >> > From: Jarek Potiuk <ja...@potiuk.com> >> > Sent: Wednesday, June 25, 2025 10:44 AM >> > To: dev@airflow.apache.org >> > Subject: [DISCUSS] Switching base Python container images to >> > python-standalone ? >> > >> > Hello here, >> > >> > Together with Aritra we are looking into adding a few more things to >> > our images (golang tool chain for CI image), also Shahar is >> > experimenting with Rust tool chain and I also recently realized (by >> > some of the issues we had) that 'Docker Official Python Image' that is >> > part of 'Official Program' [1] is not as 'Official' as I thought so we >> > discuss about changing the base of our images (first CI and then when >> > we see it works fine - PROD) >> > >> > Currently we are using the 'Official' image - but after some issues >> > and discussions with people at PyCon and FOSS Backstage (I had a >> > chance to talk to Python maintainers and even had a few beers with >> > them) - it turned out that the official Python Image' is maintained by >> > 'a community's which really is a few pretty random people - and that >> > explains for example why we have sometimes unpacked security >> > vulnerabilities in setuptools etc. - because they made some >> > compatibility choices and decisions that do not allow them to upgrade >> > easily, also they had some delays in releasing updated Python >> > versions. And Docker does not **really** do much vetting there. >> > >> > So I think it would be good to switch how we build the base for our >> images. >> > And following the experience of `uv python` [2] - it seems that maybe >> > using "python-standalone" [3] project is a good alternative. It's >> > managed by Astral now (so yes - another dependency on them), but what >> > you have with it you have practically 100% complete Python interpreter >> > installed in seconds. >> > We could continue using debian-slim as a "base, base image" - and >> > install python using "python-standalone". There are a few >> > incompatibilities [4] of the distributions of Python, but there are >> > very few and mostly related to some obscure systems (compatibilities >> > with terminal in REPL, and gtk / UI integration that is anyhow not >> > really working in "standard" Python distributions). >> > >> > I would love to hear what you think - happy to get any feedback/ >> > insights, suggestions and answer additional questions, provide some >> > links to past "troubles" we had with Python "Official" images etc. >> > >> > J. >> > >> > >> > [1] Official Python Images - https://hub.docker.com/_/python [2] UV >> > Python installation - https://docs.astral.sh/uv/guides/install-python/ >> > [3] Python Standalone project - >> > https://github.com/astral-sh/python-build-standalone >> > [4] Python Standalone incompatibilities - >> > https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html >> > ________________________________ >> > Strike Technologies, LLC (“Strike”) is part of the GTS family of >> > companies. Strike is a technology solutions provider, and is not a >> > broker or dealer and does not transact any securities related business >> > directly whatsoever. This communication is the property of Strike and >> > its affiliates, and does not constitute an offer to sell or the >> > solicitation of an offer to buy any security in any jurisdiction. It >> > is intended only for the person to whom it is addressed and may >> > contain information that is privileged, confidential, or otherwise >> protected from disclosure. >> > Distribution or copying of this communication, or the information >> > contained herein, by anyone other than the intended recipient is >> > prohibited. If you have received this communication in error, please >> > immediately notify Strike at i...@striketechnologies.com, and delete >> and destroy any copies hereof. >> > ________________________________ >> > >> > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any >> > attachments are intended solely for the addressee. This transmission >> > is covered by the Electronic Communications Privacy Act, 18 U.S.C >> > ''2510-2521. The information contained in this transmission is >> > confidential in nature and protected from further use or disclosure >> > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be >> > subject to attorney-client or other legal privilege. Your use or >> > disclosure of this information for any purpose other than that >> > intended by its transmittal is strictly prohibited, and may subject >> > you to fines and/or penalties under federal and state law. If you are >> > not the intended recipient of this transmission, please DESTROY ALL >> > COPIES RECEIVED and confirm destruction to the sender via return >> transmittal. >> > >> ________________________________ >> Strike Technologies, LLC (“Strike”) is part of the GTS family of >> companies. Strike is a technology solutions provider, and is not a broker >> or dealer and does not transact any securities related business directly >> whatsoever. This communication is the property of Strike and its >> affiliates, and does not constitute an offer to sell or the solicitation of >> an offer to buy any security in any jurisdiction. It is intended only for >> the person to whom it is addressed and may contain information that is >> privileged, confidential, or otherwise protected from disclosure. >> Distribution or copying of this communication, or the information contained >> herein, by anyone other than the intended recipient is prohibited. If you >> have received this communication in error, please immediately notify Strike >> at i...@striketechnologies.com, and delete and destroy any copies hereof. >> ________________________________ >> >> CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments >> are intended solely for the addressee. This transmission is covered by the >> Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The >> information contained in this transmission is confidential in nature and >> protected from further use or disclosure under U.S. Pub. L. 106-102, 113 >> U.S. Stat. 1338 (1999), and may be subject to attorney-client or other >> legal privilege. Your use or disclosure of this information for any purpose >> other than that intended by its transmittal is strictly prohibited, and may >> subject you to fines and/or penalties under federal and state law. If you >> are not the intended recipient of this transmission, please DESTROY ALL >> COPIES RECEIVED and confirm destruction to the sender via return >> transmittal. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >> For additional commands, e-mail: dev-h...@airflow.apache.org >> >
