jason810496 opened a new pull request, #74341:
URL: https://github.com/apache/airflow/pull/74341
- related: #73723, #74249, #74036
- related: ADR-0015
(`airflow-core/adr/lang-sdk/0015-per-dag-source-in-bundle-artifact.md`)
- **next**: #74342
## Why
`ExecutableCoordinator` had no Dag importer, so Airflow could not parse
native Go Dags, and an executable bundle embedded a single source file for all
of its Dags.
## How
- An executable bundle now embeds one source file per native Dag plus the
entrypoint, back to back in the existing source region. The `AFBNDL01` trailer
is unchanged (`footer_ver = 1`), and the manifest indexes the files:
```yaml
entrypoint_path: "example/bundle/main.go"
dag_source_paths:
orders: "example/bundle/main.go"
reports: "example/bundle/dags/reports.go"
sources:
- {path: "example/bundle/main.go", offset: 0, length: 1532, sha256:
"..."}
- {path: "example/bundle/dags/reports.go", offset: 1532, length: 811,
sha256: "..."}
```
- `ExecutableDagImporter` claims a file by its `AFBNDL01` trailer, not its
name, so a Go binary with no extension is parsed.
- `get_source_code(definition, dag_id=None)` follows #74249: a Dag's own
file, the entrypoint for an unmapped Dag, and a notice when the bundle embeds
no source.
- As in #74036, `safe_mode` does not apply and every bundle is parsed,
including bundles that only register task handlers.
## What
- `executable/_bundle_reader.py`: `read_bundle_source`,
`read_bundle_entrypoint_source` and `read_bundle_language`. Each region's
bounds and sha256 are checked.
- `ExecutableCoordinator`: opens and verifies a bundle once for its metadata
and sources, and builds the parse command `[<bundle>]` with the bundle's
supervisor schema version.
- `ExecutableDagImporter`, registered in `COORDINATOR_DAG_IMPORTERS`.
- `executable-bundle-spec.rst` and `airflow-metadata.schema.json`: `source`
is replaced by `entrypoint_path`, `dag_source_paths` and `sources`.
- `go.rst`: how native Go Dags are parsed. Until the Go runtime answers the
Dag-parse request, every bundle binary records an import error, so the page
says to list task-handler-only binaries in `.airflowignore`.
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes, with help of Claude Code Opus 5.5 following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
