jason810496 opened a new pull request, #74043: URL: https://github.com/apache/airflow/pull/74043
STACKLINE related: #71929 Part of the native Dag e2e stack, split out of #74035. Builds on #74035 ([layer diff](https://github.com/apache/airflow/compare/jason/lang-sdk-e2e/03-native-dag-parse...jason/lang-sdk-e2e/03e-lang-sdk-dag-bag)). ## Why CLI commands and `airflow dags reserialize` load Dags through a Dag bag, which cannot parse a native Lang-SDK Dag file yet. ## What changes - `CoordinatorDagImporter.import_definition` runs the same `LangSDKDagFileProcessorProcess` as the Dag processor, with the same fill and validation, and returns each Dag as a `SerializedLangSDKDAG`, the `SerializedDAG` the scheduler loads. A Dag bag has no API client, so a runtime request other than the parse result or `MaskSecret` gets an error. - A Dag bag holds `BaggedDAG = DAG | SerializedLangSDKDAG`. It keeps its duplicate id check for a native Dag and skips the SDK-only checks and cluster policies. - `sync_bag_to_db` does not store a native Dag: the Dag processor does. - `airflow dags test`, `tasks test`, `tasks render` and `tasks list` refuse a native Dag, and the Python processor refuses its callbacks. - The recorded TypeScript and Java runtime payloads must pass `validate_serialized_dag`. ```bash airflow dags test typescript_native_example # SystemExit: Dag 'typescript_native_example' is a native Lang-SDK Dag, which this command cannot run. ``` ## Deviations from #71929 - A Dag bag holds a native Dag as a `SerializedDAG`, not an `airflow.sdk.DAG`. ## How to test ```bash uv run --project airflow-core pytest airflow-core/tests/unit/dag_processing/test_dagbag.py airflow-core/tests/unit/dag_processing/test_processor.py airflow-core/tests/unit/utils/test_cli_util.py -xvs uv run --project task-sdk pytest task-sdk/tests/task_sdk/coordinators/test_dag_importer.py -xvs ``` --- ##### Was generative AI tooling used to co-author this PR? - [x] Yes, with help of Claude Code Opus 5.5 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
