dilnazanlid opened a new pull request, #73457: URL: https://github.com/apache/airflow/pull/73457
This PR introduces **ADR-0001** to document the process model and supervision interfaces for **AIP-85: Dag Importer**. Before finalizing the core Dag processor implementation, this ADR captures key architectural decisions and open design questions: 1. **Process ownership moves to the importer:** The manager supervises imports via a non-blocking `ImportHandle` protocol returned by `start_import()`. Static formats (JSON/YAML) parse inline without forking, while code execution (Python, JVM) isolates itself. 2. **Unit of work is the Dag definition:** Each definition inside a bundle or container (e.g., zip members) is queued and isolated independently to prevent cross-Dag import side effects. 3. **Garbage collection & memory hygiene:** `@final fork_import()` transparently manages `gc.disable()` / `gc.enable()` across forks, maintaining the manager’s startup `gc.freeze()`. 4. **Importer lifecycle & pre-warming:** Importers share the lifetime of the manager; `warm_up()` amortizes runtime/classpath initialization before heap freezing. 5. **Reconstructible wire envelopes:** `DagDefinitionEnvelope` passes definitions across process boundaries as coordinates rather than file contents. 6. **Open questions for discussion:** Highlights three design questions (serialization placement for inline imports, non-Python callback execution, and Lang-SDK coordinator launcher sharing). Related to: [AIP-85](https://cwiki.apache.org/confluence/x/_Q7OEg) --- ##### Was generative AI tooling used to co-author this PR? - [x] Yes — Claude Code (Opus 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
