zozo123 opened a new pull request, #73961: URL: https://github.com/apache/airflow/pull/73961
Adds `OpenShellSandboxBackend`, a self-hosted `SandboxToolset` backend that runs sandboxes through an [NVIDIA OpenShell](https://github.com/NVIDIA/OpenShell) gateway (Docker, Podman or Kubernetes), behind a new `openshell` extra. Not attachable in this version: `SandboxSpec.owner` is refused, as on OpenSandbox. ### Why another backend OpenShell is self-hosted and Apache-2.0, so the system test runs on a laptop with no paid account. Its runtime enforces the toolset's defaults natively: the workload container has no network interface, Landlock and seccomp confine it, and a per-sandbox supervisor opens every connection against a policy, so deny-all and a hostname allowlist on port 443 are enforced by the runtime, not approximated. OpenSandbox also offers self-hosted deny-all with a read-back; what differs here is that no DNS query leaves a deny-all sandbox, and that the backend re-reads the effective policy before and after every command, not only at create, because OpenShell policies can be widened on a running sandbox (by an admin, a gateway-wide policy, or auto-approved proposals). That was measured below. ### What it does - **Egress:** a default spec sends a policy with no egress rule. `allow_egress_to` becomes one rule per host, port 443, any binary. Refused: `block_network=False`, `allow_egress_to_cidrs`, and non-hostname entries. - **Fail-closed:** after create, the effective policy is read back via `GetSandboxConfig`. The sandbox is destroyed and the task fails unless the policy admits exactly the requested hosts, comes from the sandbox rather than a GLOBAL policy, keeps Landlock at `hard_requirement`, and has neither `proposal_approval_mode=auto` nor agent proposals enabled. The same check runs before and after every command. - **Commands:** OpenShell's exec timeout leaves the process running (NVIDIA/OpenShell#3159, closed not_planned), so a guest wrapper runs the command from stdin in its own session, spools output to `/tmp`, and on timeout SIGKILLs the whole session, rescanning until empty. The exit status comes from the wrapper's own process status, never from a trailer the command could print. Each stream is capped on the worker. If the gateway goes quiet past the budget + 30 s, the sandbox is destroyed. A gateway restart is reported as an unknown outcome, never a success. - **Files:** content goes over stdin in 768 KiB chunks (the gateway caps an argument at 32 KiB and a request at 1 MiB). Reads are raw bytes, capped in the guest, and an oversized read reports its real size. - **Env:** the 16 proxy and CA variables the supervisor silently removes or overwrites, and `OPENSHELL_*`, are refused. Everything else reaches commands as given (no login shell). - **Credentials:** ambient, from the OpenShell CLI's gateway registration; no connection type (following https://github.com/apache/airflow/pull/71672#discussion_r4042149576). - **Extra:** `openshell>=0.1.2,<0.2` plus `grpcio>=1.78.0` and `protobuf>=6.31.1`, all behind `python_version >= "3.11"`. The wheel declares lower floors, but its generated code fails to import below these (measured). A temporary exclude-newer override covers the 0.1.2 release date and can be dropped from 2026-10-03. ### Measured against a live OpenShell 0.1.2 gateway (Docker driver on colima, `python:3.12-slim`) | Check | Result | |---|---| | Create, deny-all | 1.05–1.64 s; per-command overhead including both policy reads 18–47 ms | | `sleep 300`, 3 s budget | timed out at 3.04 s; no `sleep 300` left in `/proc` | | Background children, forking tree, busy loop, on timeout | all killed | | `sleep 120 & echo started` | returns in 0.03 s; the child keeps running | | 20 MB stdout, and 20 MB written straight into the exec stream | cut to the 50 KiB cap; worker peak about 165 KB | | 100 KB binary / 10 MB round trips | exact sha256; 10 MB read with a 5 MiB limit reports 10,000,000 bytes | | Deny-all | TCP to IP and hostname fails with EACCES; DNS answers a synthetic 198.18.0.3 | | Allowlist `example.com` | HTTPS 200; port 80 and other hosts denied | | GLOBAL policy / auto-approval / admin widening | refused at create, or detected on the next command; sandbox destroyed | | Denied connect auto-approved under global auto mode | policy widened after 10.2 s; next command failed the task | | Gateway `docker restart` mid-command | recoverable "may or may not have run"; next command works; files kept | | Gateway paused mid-command | sandbox destroyed; `timed_out` + `sandbox_terminated` at 46.6 s | ### Limitations - Python 3.11+ only. The SDK is Alpha; the backend uses its private `_stub` for exec and config reads (the public exec buffers all output), hence the `<0.2` cap. - Policy verification detects a widening after the fact; it does not prevent one. A command already running when the policy is widened can use it. On Kubernetes, egress also depends on a NetworkPolicy the cluster's CNI must enforce, which the check cannot see. Only the Docker driver was measured. - Without OIDC, an mTLS client is a gateway-wide admin, and the certificates from `generate-certs` effectively never expire; use your own PKI. - DNS returns synthetic 198.18.0.0/15 answers instead of failing. Allowlisted HTTPS is decrypted and re-encrypted by the supervisor with its own CA, so clients with their own trust store need pointing at it. Every denied connect becomes a draft proposal visible to operators. - No server-side lifetime: a dead worker leaks its sandbox. Sandboxes are labelled `created-by=airflow` and `airflow-created-at`, and the docs include a reaper recipe. - A child the command starts with `setsid` itself survives a timeout. Output is sent only when a command ends, so proxies in front of the gateway need an idle timeout of at least the budget + 30 s. Writes over 768 KiB are not atomic. BusyBox images are unsupported. ### Tests - Unit: `test_openshell.py`, 89 passed on Linux (84 on macOS; the 5 real-`/proc` wrapper tests are Linux-only). - System: `example_sandbox_toolset_openshell.py` passed against the live gateway. - ruff, mypy and prek are clean. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
