Hi Yossi, I agree with Vikram and Kaxil that this requires a more comprehensive discussion on the devlist rather than on GitHub. Given the scope of this feature, it likely requires an Airflow Improvement Proposal (AIP).
While the design notes and reference implementation are a helpful starting point, proposing a "ready" solution before community engagement is not the standard process for Apache Airflow, nor is it how community contributions typically succeed. We need to understand the reasoning and thought process behind the idea, interact with the author, and likely see a formal demo or presentation at a dev call after the initial discussion when different ideas and thoughts from the community members are shared. This ensures the community is involved in designing the best approach and that the solution aligns with other ongoing projects and that community accepts it as "their" as much as "your" proposal - because no matter if you commit to maintaining it, the moment we accept it, it is "ours" (community) to maintain - not "yours". The discussion on the devlist and the subsequent AIP should address several key areas: - Understanding "whys" and "whats" - before we go to "hows" - The core goals and typical use cases. - Interaction with the common.ai provider. - The reasoning behind selecting these specific sandboxes, and allowing more options rather than choosing just one, for example. - Alternative implementation methods. - Threat models and security properties provided by the solution (very important for handling, reporting, and communicating all kinds of security issues to our users). Your current PR and implementation will be treated as one potential solution - and mostly a starting point - among others considered during the discussion. Please be prepared for the community to suggest or reach a consensus on design choices that may differ from your initial proposal. Regarding the involvement of CEOs from other sandboxing solutions, they are more than welcome to join the conversation. However, their primary role should be helping to shape the scope and design of the solution rather than simply contributing specific modules. A GitHub discussion is insufficient for a feature of this magnitude, as it has significant implications for both existing and planned Airflow components. I look forward to continuing this discussion here on the devlist and resulting AIP. Best regards, Jarek Potiuk On Thu, Jun 25, 2026 at 8:46 AM Yossi Eliaz via dev <[email protected]> wrote: > Hi everyone, > > Thank you for the feedback. I would love to continue this discussion on > GitHub. We are willing to maintain the sandbox wrapper as well as the > support for islo as one of the sandbox providers. Additionally, I will > reach out to the CEOs of Modal, Daytona, E2B, and Tensorlake to encourage > them to contribute their respective modules. > > I think this integration could be very exciting for data engineering and > workflows in general. > > Best, > Yossi > > On Wed, Jun 24, 2026 at 5:52 PM Vikram Koka <[email protected]> wrote: > > > IRONSCALES couldn't recognize this email as this is the first time you > > received an email from this sender vikram @ astronomer.io > > > > Thanks for the proposal, Yossi. > > > > +1 to Kaxil's comments above. > > We discussed this offline earlier in the context of Agentic workflows and > > therefore interaction with the common.ai provider would be the logical > > first step here. > > > > Best regards, > > Vikram > > > > > > On Wed, Jun 24, 2026 at 6:37 AM Kaxil Naik <[email protected]> wrote: > > > >> Thanks for sending the Proposal Yossi. I haven't looked it in detail yet > >> but first thing would be to figure out the interaction with common.ai > >> provider since the primary motivation on your PR is AI Agents/LLM > >> > >> On Wed, 24 Jun 2026 at 14:32, Yossi Eliaz via dev < > [email protected] > >> > > >> wrote: > >> > >> > Hi all, > >> > > >> > I’d like to propose a new Sandbox provider for running Airflow tasks > in > >> > fast, isolated, ephemeral cloud sandboxes using Daytona, E2B, Modal, > and > >> > islo backends. > >> > > >> > The goal is to make code safer to run: credentials are injected only > >> into > >> > the disposable sandbox, not the Airflow worker, while keeping tasks > >> > scalable and isolated from one another. > >> > > >> > The implementation includes: > >> > > >> > - SandboxOperator and @task.sandbox for running commands in a sandbox > >> > - SandboxExecutor for routing tasks through sandboxes > >> > - No core or scheduler changes; it uses the public BaseExecutor > >> interface, > >> > so I do not believe an AIP is required > >> > > >> > > >> > - PR: https://github.com/apache/airflow/pull/68847 > >> > - Issue: https://github.com/apache/airflow/issues/68845 > >> > - Design notes: https://github.com/zozo123/airflow-provider-sandbox > >> > > >> > This work was developed with assistance from Claude, but all code has > >> been > >> > human-reviewed. > >> > > >> > Best, > >> > Yossi > >> > > >> > > >
