jason810496 opened a new issue, #69905: URL: https://github.com/apache/airflow/issues/69905
### Background #69295 adds `airflow-ts-pack`, which embeds the bundle manifest as a single first-line comment in `bundle.mjs` (`//# airflowMetadata=<base64 YAML>`); the Node coordinator reads only that head line (bounded `readline()`, 1 MiB cap). The Go SDK's AFBNDL01 executable bundle format provides two things the TS format does not (see the Trailer Layout section of `task-sdk/docs/executable-bundle-spec.rst`): - **Integrity validation** — a SHA-256 digest over the code region, verified by the scanner before the bundle is used. - **An embedded source region** — the original entrypoint (`.ts` / `.js`), so the UI can display the code; needed for the native TypeScript Dag feature (#69288). Review on #69295 agreed to keep that PR to the packing flow and refine the packing spec in a follow-up. Since `bundle.mjs` must remain directly runnable by `node`, the structure cannot be a binary EOF trailer like AFBNDL01; instead the head comment can carry the layout — a format version, start/end offsets for each section, and the digest. ### What needs to happen 1. Specify a versioned head-comment layout for `.mjs` bundles recording section offsets (bundled code, embedded source, metadata) and a SHA-256 digest of the code region, documented alongside AFBNDL01 in `task-sdk/docs/executable-bundle-spec.rst`. 2. Update `airflow-ts-pack` (`ts-sdk/src/cli/pack.ts`) to emit the new layout and embed the original entrypoint source. 3. Update the Node coordinator (`task-sdk/src/airflow/sdk/coordinators/node/coordinator.py`) to verify the digest during scan — log and skip on mismatch, with the same `(path, inode, mtime, size)` result caching as the AFBNDL01 reader algorithm — and expose the embedded source. ### Acceptance criteria - A packed `bundle.mjs` remains directly runnable with `node bundle.mjs`. - The coordinator logs and skips a truncated or modified bundle on digest mismatch. - The embedded entrypoint source is retrievable for display. - `executable-bundle-spec.rst` documents the TS layout with its own format version. ### Context - Originating PR: #69295 - Review threads: metadata placement (trailer moved to head): https://github.com/apache/airflow/pull/69295#discussion_r3520504501 ; deferred spec refinement ("we can refine the packing spec in the follow-up"): https://github.com/apache/airflow/pull/69295#discussion_r3551043871 - Go parallel: AFBNDL01 trailer layout, `task-sdk/docs/executable-bundle-spec.rst` - Related: #69288 (TypeScript SDK: native TypeScript Dag declaration — the source-viewing consumer) --- Drafted-by: Claude Code (Fable 5); reviewed by @jason810496 before posting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
