+1 On Tue, May 26, 2026 at 11:22 AM Danny McCormick via dev < [email protected]> wrote:
> +1 - we generally assume Beam workers are containerized/single tenant, so > this seems consistent. > > Thanks, > Danny > > On Tue, May 26, 2026 at 2:13 PM Shunping Huang <[email protected]> > wrote: > >> Hi everyone, >> >> I am writing to propose that we disable build isolation in boot.go of the >> Beam Python Containers by default, starting with the upcoming Beam 2.75 >> release. >> >> *Context* >> >> - Starting with pip version 25.3 ( >> https://discuss.python.org/t/announcement-pip-25-3-release/104550), >> build isolation became the default mechanism for package installation. >> This >> means pip now creates a *temporary, isolated* environment to build >> packages, which often requires fetching build-time dependencies (like >> setuptools) from external repositories. >> - In restricted network environments—such as Dataflow jobs configured >> without public IPs—this change can cause pipelines to hang or fail with >> "network unreachable" errors during container initialization. We have >> observed these failures in use cases where upgrading to a newer image >> version (containing a more recent pip) broke previously working pipelines. >> - To address this, we already introduced an experimental flag in Beam >> 2.72: `--experiments=pip_no_build_isolation` ( >> https://github.com/apache/beam/pull/37331). This flag instructs the >> Beam SDK's container boot process to disable build isolation during >> package >> installation, preventing external network calls. >> >> *Proposal* >> I propose making this disabled state the default behavior for Beam 2.75 >> (Beam 2.74 is currently in RC). Here are some rationales: >> >> - Before upgrading to pip 25.3, Beam did not use build isolation, and >> users rarely reported issues regarding package builds. >> - Build isolation requires network access, which fundamentally limits >> the ability to run pipelines in secure or restricted network environments >> (See the above Dataflow runner example). Disabling it by default removes >> our reliance on external networks and PyPI availability, especially for >> users who already include their dependencies in custom containers. >> - To accommodate any edge cases, we can introduce a new flag >> (`--experiments=pip_use_build_isolation`) to turn build isolation back on >> if users truly need it, though we have not yet encountered a use case >> requiring it. >> >> >> Please let me know your thoughts or if you have any concerns. >> >> Thanks, >> >> Shunping >> >
