This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a commit to branch v3-4-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/v3-4-test by this push:
new 0873fcdab3b [v3-4-test] TS SDK: Bump to 1.0.0-beta1 and align README
with the guide (#74693) (#75300)
0873fcdab3b is described below
commit 0873fcdab3baab2aa47e0a60ec000dbb7ff42fed
Author: github-actions[bot]
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Thu Oct 22 20:41:03 2026 +0800
[v3-4-test] TS SDK: Bump to 1.0.0-beta1 and align README with the guide
(#74693) (#75300)
* TS SDK: Bump to 1.0.0-beta1 and align README with the guide
Move the human-in-the-loop docs into the TypeScript SDK guide, describe the
TaskClient methods in a table, and trim the README and the API reference
landing page to a quick start that links to the guide.
* Keep the HITL params warning in the TypeScript SDK guide
---------
(cherry picked from commit f3cf550b94b0f3794e6d6aa7ea57d71312b38a86)
Co-authored-by: Jason(Zhe-You) Liu
<[email protected]>
---
.../language-sdks/typescript.rst | 91 +++++--
kubernetes-tests/lang_sdk/ts_example/package.json | 2 +-
task-sdk/docs/ts-bundle-spec.rst | 2 +-
ts-sdk/DEVELOPMENT.md | 14 +-
ts-sdk/README.md | 268 +++------------------
ts-sdk/docs/index.md | 43 ++--
ts-sdk/docs/package-lock.json | 4 +-
ts-sdk/docs/package.json | 2 +-
ts-sdk/docs/tests/deployment-checks.test.mjs | 4 +-
ts-sdk/example/README.md | 80 +++---
ts-sdk/example/package.json | 2 +-
ts-sdk/package.json | 2 +-
12 files changed, 186 insertions(+), 328 deletions(-)
diff --git
a/airflow-core/docs/authoring-and-scheduling/language-sdks/typescript.rst
b/airflow-core/docs/authoring-and-scheduling/language-sdks/typescript.rst
index 84b40c60ca9..5b2ca7a816f 100644
--- a/airflow-core/docs/authoring-and-scheduling/language-sdks/typescript.rst
+++ b/airflow-core/docs/authoring-and-scheduling/language-sdks/typescript.rst
@@ -527,22 +527,36 @@ has retries.
The ``TaskClient``
~~~~~~~~~~~~~~~~~~
-* ``getVariable(key)`` returns the Variable as a string, or ``null`` when it
does not exist.
- ``getVariableOrThrow(key)`` throws ``VariableNotFoundError`` instead, like
Python's ``Variable.get`` without
- a default.
-* ``setVariable(key, value, description?)`` stores a Variable, replacing any
existing value, and
- ``deleteVariable(key)`` removes one. Values are strings, so serialize
structured data, for example with
- ``JSON.stringify``, before storing it.
-* ``getConnection(connId)`` returns a ``ConnectionResult`` with ``id`` and
``type``, plus ``host``, ``schema``,
- ``login``, ``password``, ``port`` and ``extra``, each of which may be
missing or ``null``. It returns ``null``
- when the connection does not exist, and ``getConnectionOrThrow(connId)``
throws ``ConnectionNotFoundError``
- instead, like Python's ``BaseHook.get_connection``.
-* ``getXCom<T>({ key, ... })`` reads an XCom value, or ``null`` when it does
not exist. ``dagId``, ``runId``,
- ``taskId`` and ``mapIndex`` default to the current task; pass ``taskId`` to
read an upstream task's XCom.
- See :ref:`typescript-sdk/types` for how values map to JavaScript types.
-* ``setXCom({ key, value, ... })`` publishes an XCom value.
-* ``taskStateStore`` is this task instance's key/value store, with
``get<T>(key)``,
- ``set(key, value, { retentionMs? })``, ``delete(key)`` and ``clear()``.
+.. list-table::
+ :header-rows: 1
+ :widths: 40 60
+
+ * - Method
+ - Description
+ * - ``getVariable(key)``
+ - The Variable as a string, or ``null`` when it does not exist.
+ * - ``getVariableOrThrow(key)``
+ - Same, but throws ``VariableNotFoundError``, like Python's
``Variable.get`` without a default.
+ * - ``setVariable(key, value, description?)``
+ - Stores a Variable, replacing any existing value. Values are strings, so
serialize structured data, for
+ example with ``JSON.stringify``.
+ * - ``deleteVariable(key)``
+ - Removes a Variable.
+ * - ``getConnection(connId)``
+ - A ``ConnectionResult`` with ``id`` and ``type``, plus ``host``,
``schema``, ``login``, ``password``,
+ ``port`` and ``extra``, each of which may be missing or ``null``.
``null`` when the connection does not
+ exist.
+ * - ``getConnectionOrThrow(connId)``
+ - Same, but throws ``ConnectionNotFoundError``, like Python's
``BaseHook.get_connection``.
+ * - ``getXCom<T>({ key, ... })``
+ - An XCom value, or ``null`` when it does not exist. ``dagId``,
``runId``, ``taskId`` and ``mapIndex``
+ default to the current task; pass ``taskId`` to read an upstream task's
XCom. See
+ :ref:`typescript-sdk/types`.
+ * - ``setXCom({ key, value, ... })``
+ - Publishes an XCom value.
+ * - ``taskStateStore``
+ - This task instance's key/value store, with ``get<T>(key)``, ``set(key,
value, { retentionMs? })``,
+ ``delete(key)`` and ``clear()``. See
:ref:`typescript-sdk/task-state-store`.
.. note::
@@ -550,6 +564,8 @@ The ``TaskClient``
stored value when the Variable is read back. ``setVariable`` without a
description clears the existing
description, and ``deleteVariable`` succeeds even when the Variable does
not exist.
+.. _typescript-sdk/task-state-store:
+
Task state store
----------------
@@ -647,6 +663,49 @@ XComs are stored as JSON, so a value read with ``getXCom``
arrives as the matchi
JavaScript has a single ``number`` type, so integers and decimals arrive as
the same type, and an integer
larger than ``Number.MAX_SAFE_INTEGER`` (2\ :sup:`53` − 1) may lose
precision.
+Human-in-the-loop tasks
+-----------------------
+
+``approval`` and ``hitl``, from ``apache-airflow-ts-sdk/hitl``, make a task
that waits for a person, as Python's
+``ApprovalOperator`` and ``HITLOperator`` do. Pass one to ``dag.task`` in
place of a handler. Options are the
+Python keyword arguments in camelCase:
+
+.. code-block:: typescript
+
+ import { approval } from "apache-airflow-ts-sdk/hitl";
+
+ const signOff = dag.task(
+ "sign_off",
+ approval({
+ subject: ({ report }: { report: { version: string } }) => `Ship
${report.version}?`,
+ responseTimeout: 4 * 60 * 60, // seconds
+ defaults: "Reject",
+ params: { replicas: { value: 3, schema: { type: "integer", minimum: 1
} } },
+ }),
+ )({ report });
+
+ signOff.before(published);
+
+The task waits in ``awaiting_input`` without holding a worker slot, and
resumes once someone responds on the
+**Required Actions** page or ``responseTimeout`` passes.
+
+* ``subject`` and ``body`` (Markdown) are a string, or a function of the
task's inputs.
+* ``responseTimeout`` is in seconds. When it passes, ``defaults`` is applied,
or the task fails without one.
+ ``executionTimeout`` is rejected on a HITL task.
+* ``params`` adds form fields, each with a ``value`` that pre-fills it, and an
optional ``description`` and
+ JSON Schema. Answers arrive as ``paramsInput``. An answer that fills in some
params but not exactly the declared ones
+ fails the task, as ``HITLOperator`` does. The values are not yet checked
against their schema when the task resumes,
+ so an answer given through the REST API can fall outside it.
+* ``assignedUsers`` limits who can respond.
+* The task returns ``{ chosenOptions, paramsInput, respondedAt,
respondedByUser, timedout }``.
+
+On ``"Reject"``, ``approval`` succeeds and skips the tasks directly
downstream, as ``ApprovalOperator`` does.
+Set ``failOnReject: true`` to fail instead, or ``ignoreDownstreamTriggerRules:
true`` to skip everything
+downstream. ``hitl`` takes any ``options``, with ``multiple`` for several, and
never skips or fails on the answer.
+
+A retry or a cleared task writes a fresh request. Branching on the response
and notifiers are not supported yet.
+``ts-sdk/example/src/hitl.ts`` has one Dag per human-in-the-loop feature.
+
.. _typescript-sdk/build:
Building a bundle
diff --git a/kubernetes-tests/lang_sdk/ts_example/package.json
b/kubernetes-tests/lang_sdk/ts_example/package.json
index f6cfe124c40..94290747263 100644
--- a/kubernetes-tests/lang_sdk/ts_example/package.json
+++ b/kubernetes-tests/lang_sdk/ts_example/package.json
@@ -1,7 +1,7 @@
{
"name": "apache-airflow-lang-sdk-k8s-ts-example",
"private": true,
- "version": "0.1.0-beta1",
+ "version": "1.0.0-beta1",
"packageManager": "[email protected]",
"type": "module",
"license": "Apache-2.0",
diff --git a/task-sdk/docs/ts-bundle-spec.rst b/task-sdk/docs/ts-bundle-spec.rst
index 03ee63ceea0..4e4ddc7888f 100644
--- a/task-sdk/docs/ts-bundle-spec.rst
+++ b/task-sdk/docs/ts-bundle-spec.rst
@@ -127,7 +127,7 @@ The ``airflowMetadata`` payload is compact UTF-8 JSON with
this logical shape:
"airflow_bundle_metadata_version": "1.0",
"sdk": {
"language": "typescript",
- "version": "0.1.0-beta1",
+ "version": "1.0.0-beta1",
"supervisor_schema_version": "2026-06-16"
},
"source": "main.ts",
diff --git a/ts-sdk/DEVELOPMENT.md b/ts-sdk/DEVELOPMENT.md
index 1966842c31b..ac8ed73eb4e 100644
--- a/ts-sdk/DEVELOPMENT.md
+++ b/ts-sdk/DEVELOPMENT.md
@@ -148,16 +148,16 @@ names the source commit that produced the tarball. The
tag must already exist
on `apache/airflow` before the dispatch below can reference it as `--ref`:
```bash
-git tag ts-sdk/0.1.0-beta1 <commit-sha>
-git push upstream ts-sdk/0.1.0-beta1
+git tag ts-sdk/1.0.0-beta1 <commit-sha>
+git push upstream ts-sdk/1.0.0-beta1
```
To submit the package to npm's private staging area for review, run:
```bash
-gh workflow run ts-sdk-release.yml --repo apache/airflow --ref
ts-sdk/0.1.0-beta1 \
+gh workflow run ts-sdk-release.yml --repo apache/airflow --ref
ts-sdk/1.0.0-beta1 \
-f release_type=staged \
- -f tag=ts-sdk/0.1.0-beta1 \
+ -f tag=ts-sdk/1.0.0-beta1 \
-f npm_tag=beta
```
@@ -181,14 +181,14 @@ that is already staged; approve or reject that staged
version instead.
To publish directly without npm's staging review, trigger the formal path:
```bash
-gh workflow run ts-sdk-release.yml --repo apache/airflow --ref
ts-sdk/0.1.0-beta1 \
+gh workflow run ts-sdk-release.yml --repo apache/airflow --ref
ts-sdk/1.0.0-beta1 \
-f release_type=formal \
- -f tag=ts-sdk/0.1.0-beta1 \
+ -f tag=ts-sdk/1.0.0-beta1 \
-f npm_tag=beta
```
Use `latest` for stable releases. A prerelease may only use the channel derived
-from its own prerelease identifier (`0.1.0-beta1` → `beta`) or `next` —
+from its own prerelease identifier (`1.0.0-beta1` → `beta`) or `next` —
`scripts/validate-release-inputs.mjs` rejects any other tag. The workflow also
rejects a dispatch that would move the selected npm dist-tag backward as of
when it runs. For a staged release, the dist-tag only actually moves later, at
diff --git a/ts-sdk/README.md b/ts-sdk/README.md
index 10db9aaa7f5..33f8178ea22 100644
--- a/ts-sdk/README.md
+++ b/ts-sdk/README.md
@@ -19,149 +19,44 @@
# Airflow TypeScript SDK
-Write Apache Airflow Dags and tasks in TypeScript.
+Write Apache Airflow Dags in TypeScript, and run them on Node.js.
-**Status:** 0.1.0-beta1 · API may change · Node 22+ · ESM-only
-
-There are two ways to use it:
-
-- **Dag definition**: declare the whole Dag in TypeScript, with its schedule,
tasks, options and dependencies.
-- **Python Dag with stub TaskHandler**: a TypeScript `TaskHandler` implements
each stub task of a Python Dag.
-
-Both use the same task API, and one bundle can serve both.
+**Status:** 1.0.0-beta1 · API may change · Node 22+ · ESM-only
## Installation
```bash
-npm install [email protected]
+npm install [email protected]
npm install --save-dev esbuild
```
-## Declaring a Dag
+## Quick start
+
+Declare a Dag in `src/main.ts`:
```ts
-import { Bundle, Dag, getClient } from "apache-airflow-ts-sdk";
+import { Bundle, Dag } from "apache-airflow-ts-sdk";
-const dag = new Dag("sales_pipeline", { schedule: "@daily", queue:
"typescript" });
+const dag = new Dag("ts_hello", { schedule: "@daily", queue: "typescript" });
-const extract = dag.task("extract", async (): Promise<number> => {
- return Number((await getClient().getVariable("daily_row_count")) ?? "0");
+const extract = dag.task("extract", async (): Promise<number> => 42);
+const report = dag.task("report", async ({ rows }: { rows: number }) => {
+ console.log(`extracted ${rows} rows`);
});
-const transform = dag.task(
- "transform",
- async ({ rows, region }: { rows: number; region: string }) => ({ rows,
region }),
-);
-transform({ rows: extract(), region: "us" });
+report({ rows: extract() });
await new Bundle(dag).serve();
```
-A handler takes one object of named arguments, and calling a task names its
inputs. An input is either the
-reference another task's call returned, which makes this task wait for it and
receive its value, or a literal.
-
-A Dag also has order-only dependencies (`before` and `after`), task groups,
branching with `dag.if` and
-`dag.switch`, and `triggerDagRun`. See the
-[guide](https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/language-sdks/typescript.html)
-for all of them.
-
-## Implementing stub tasks of a Python Dag
-
-The Python Dag declares the tasks and their dependencies:
-
-```python
-from airflow.sdk import dag, task
-
-
-@dag
-def sales_pipeline():
- @task.stub(queue="typescript")
- def extract(): ...
-
- @task.stub(queue="typescript")
- def transform(extracted): ...
-
- transform(extract())
-
-
-sales_pipeline()
-```
-
-TypeScript implements them, with a `TaskHandler` per stub task:
-
-```ts
-import { Bundle, getClient, TaskHandler } from "apache-airflow-ts-sdk";
-
-export async function extract() {
- const rowCount = Number((await getClient().getVariable("daily_row_count"))
?? "0");
- return { rowCount };
-}
-
-export async function transform({ extracted }: { extracted: { rowCount: number
} }) {
- return { transformedRows: extracted.rowCount };
-}
-
-await new Bundle(
- new TaskHandler("sales_pipeline", "extract", extract),
- new TaskHandler("sales_pipeline", "transform", transform),
-).serve();
-```
-
-A `TaskHandler` names the `dag_id` and `task_id` it implements, so the same
`task_id` under two Dags is two
-different handlers. Nothing connects to Airflow until `bundle.serve()`, so a
unit test can build a bundle and
-call a handler through `bundle.getTaskHandler(dagId, taskId)`.
-
-### TaskFlow arguments
-
-A Python Dag that calls a stub task TaskFlow-style passes those arguments to
the handler, which destructures
-them by name:
-
-```python
[email protected](queue="typescript")
-def transform(region_code: str, threshold: float, dry_run: bool = False): ...
-
-
-transform("uk", 0.75)
-```
-
-```ts
-export async function transform({ regionCode, threshold, dryRun }:
TransformArgs) {
- // ...
-}
-```
-
-Names match ignoring case and underscores, so `region_code` reaches
`regionCode` with nothing declared on
-either side. An argument filled from another task, as in `transform(extract(),
"uk")`, arrives as that task's
-value. A dependency drawn with `>>` orders the tasks and passes nothing; read
such a value with
-`getClient().getXCom`.
-
-`withArgNames` maps a handler's name to a different Python name, when the two
cannot match on their own:
-
-```ts
-const report = withArgNames({ label: "run_label" }, async ({ summary, label }:
ReportArgs) => {
- // `label` is the call's `run_label`.
-});
-```
-
-## Writing tasks
-
-A handler is a plain, usually `async`, function. `getContext()` and
`getClient()` give it the task's context and
-Airflow access while it runs, so it takes no SDK-supplied argument. A value it
returns becomes the task's
-`return_value` XCom, and an uncaught error fails the task.
-
-## Building and deploying
-
-`airflow-ts-pack` builds the entry module and everything it imports into one
file, `dist/bundle.min.mjs`:
+Build it into a single file:
```bash
npx airflow-ts-pack src/main.ts --outdir dist
```
-- `--outdir <dir>`: output directory (default `dist`)
-- `--outfile <path>`: exact output path, whose name must end in `.min.mjs`
-
-A bundle that declares Dags goes into a Dag bundle, such as the default
`dags-folder` bundle. Route its queue to
-the Node.js coordinator:
+Put `dist/bundle.min.mjs` in a Dag bundle, such as the default `dags-folder`
bundle, and send the `typescript`
+queue to the Node.js coordinator in `airflow.cfg`:
```ini
[sdk]
@@ -169,124 +64,31 @@ coordinators = {"ts": {"classpath":
"airflow.sdk.coordinators.node.NodeCoordinat
queue_to_coordinator = {"typescript": "ts"}
```
-A bundle that only implements stub tasks goes next to the Python Dag that
declares them, or into the Dag
-bundle the coordinator's `task_handler_bundle_name` names. See the guide for
all coordinator options.
-
-See [`example/`](https://github.com/apache/airflow/tree/main/ts-sdk/example)
for a working project that serves
-both a Dag declared in TypeScript and the tasks of two Python Dags from one
bundle.
-
-## Human-in-the-loop tasks
-
-A Dag declared in TypeScript can wait for a person, as `HITLOperator` and
`ApprovalOperator` do in
-Python. `approval(...)` and `hitl(...)` come from the
`apache-airflow-ts-sdk/hitl` subpath and are
-passed to `dag.task` in place of a handler. Each is named after its Python
operator without
-"Operator", and each option is the Python keyword argument in camelCase:
-
-```ts
-import { Bundle, Dag } from "apache-airflow-ts-sdk";
-import { approval } from "apache-airflow-ts-sdk/hitl";
-
-const dag = new Dag("release", { queue: "typescript" });
-const report = dag.task("build_report", async () => ({ version: "1.4" }))();
-
-const signOff = dag.task(
- "sign_off",
- approval({
- subject: ({ report }: { report: { version: string } }) => `Ship
${report.version}?`,
- responseTimeout: 4 * 60 * 60, // seconds
- defaults: "Reject", // the answer given when the timeout passes
- }),
-)({ report });
-signOff.before(dag.task("publish", async () => {})());
-
-await new Bundle(dag).serve();
-```
-
-The task's first run writes the request and parks the task in
`awaiting_input`; its Node process
-exits, so no worker slot is held while it waits. Once someone responds on the
Required Actions page,
-or the timeout passes, the task resumes in a new process. It does not write
the request again or run
-`subject` or `body` again.
-
-- `subject` and `body` (Markdown) are a string, or a function of the task's
inputs, run once. They
- may read different inputs, and the task takes both.
-- `responseTimeout` is in whole **seconds**. With `defaults`, the defaults are
applied when it passes;
- without, the task fails, and its retries apply. A HITL task does not enforce
`executionTimeout`, so
- `dag.task` rejects it and asks for `responseTimeout` instead.
-- `assignedUsers: [{ id, name }]` limits who can respond, by the auth
manager's user id.
-- The task returns the response, which downstream tasks receive and XCom
stores under
- `return_value`: `{ chosenOptions, paramsInput, respondedAt, respondedByUser,
timedout }`.
- These are the keys of Python's `HITLOperator` result in camelCase, plus
`timedout`; `respondedAt` is an
- ISO-8601 string, and `respondedByUser` is `null` when the timeout defaults
were applied.
-
-**On "Reject", an approval succeeds and skips the tasks directly downstream by
default**, as
-`ApprovalOperator` does. The run then shows as successful, with the guarded
tasks skipped. Set
-`failOnReject: true` to fail the approval instead, or
`ignoreDownstreamTriggerRules: true` to skip
-every task downstream. `hitl(...)` offers any `options`, with `multiple` for
several, and never
-skips or fails on what is chosen, even an option named "Reject".
-
-A failed resume retries, and the retry writes a fresh request, so `approval({
failOnReject: true })`
-with `retries` asks the reviewer again on each attempt.
-
-When the task resumes, it reads its options, its `params` and its reject
options from the Dag it
-runs from, as `HITLOperator` does. A versioned Dag bundle resumes it from the
same version it
-started on; with an unversioned bundle, a bundle deployed while the request
waits is what the
-response is read against. Clearing the task starts a new attempt, which writes
a fresh request.
-
-### Forms
-
-`params` adds form fields to the request, as `HITLOperator`'s `params` does,
for `approval(...)` and
-`hitl(...)` alike. Each field is a `value` that pre-fills it, an optional
`description`, and an
-optional JSON Schema that the form renders:
-
-```ts
-import { approval } from "apache-airflow-ts-sdk/hitl";
-
-const rollout = dag.task(
- "size_rollout",
- approval({
- subject: "Approve the rollout?",
- defaults: "Reject",
- params: {
- replicas: {
- value: 3,
- description: "How many replicas to start",
- schema: { type: "integer", minimum: 1, maximum: 10 },
- },
- },
- }),
-)();
-```
-
-The answers arrive as `paramsInput`, keyed by the param names. When the
timeout applies `defaults`,
-each param's `value` is recorded as its answer. An answer that fills in some
params but not exactly
-the declared ones fails the task, as `HITLOperator` does. The values are not
yet checked against
-their schema when the task resumes, so an answer given through the REST API
can fall outside it.
-
-A `hitl(...)` whose options are exactly "Approve" and "Reject" gets the
approval buttons in the UI
-but never skips anything; use `approval(...)` when Reject should stop the
tasks after it.
-
-Branching on the response and notifiers are not supported yet.
-
-## TaskClient
+Airflow reads the bundle like any other Dag file: `ts_hello` shows up in the
UI, runs on its schedule, and
+`report` receives the value `extract` returned.
-`getClient()` returns a `TaskClient` for task-time Airflow data access, for as
long as a handler is running:
+## Learn more
-| Method |
Description |
-| --------------------------------------------------------------- |
-------------------------------------- |
-| `getVariable(key)` / `getVariableOrThrow` | Airflow
Variables |
-| `setVariable(key, value, description?)` / `deleteVariable(key)` | Variable
write / delete |
-| `getXCom(opts)` / `setXCom(opts)` | XCom
read/write |
-| `getConnection(connId)` / `getConnectionOrThrow` | Airflow
Connections |
-| `taskStateStore.get / set / delete / clear` |
Per-task-instance key/value state store |
+The [TypeScript SDK guide][guide] covers:
-Locator fields such as `dagId`, `runId`, and `taskId` default to the
-current task context when omitted.
+| Topic
| Guide section
|
+|
---------------------------------------------------------------------------------------------
| -------------------------------------------------------------- |
+| Task inputs, options, order-only dependencies, task groups, branching,
triggering another Dag | [Declaring a Dag in
TypeScript][declaring-a-dag-in-typescript] |
+| `approval` and `hitl` tasks that wait for a person
| [Human-in-the-loop tasks][human-in-the-loop-tasks]
|
+| `getContext()`, `getClient()` and the `TaskClient`
| [Writing tasks][writing-tasks]
|
+| Building a bundle with `airflow-ts-pack`
| [Building a bundle][building-a-bundle]
|
+| Where to deploy a bundle, and the `NodeCoordinator` options
| [Deploying][deploying]
|
+| What a Dag declared in TypeScript cannot do yet
| [Limitations][limitations]
|
-## Cancellation
+See [`example/`](https://github.com/apache/airflow/tree/main/ts-sdk/example)
for a working project.
-`getContext().signal` is an `AbortSignal` controlled by the active runtime.
-Pass it to `fetch()`, timers, database clients, child processes, or any other
API that accepts an abort signal,
-so tasks can clean up cooperatively when Airflow terminates the task
subprocess.
+[guide]:
https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/language-sdks/typescript.html
+[declaring-a-dag-in-typescript]:
https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/language-sdks/typescript.html#declaring-a-dag-in-typescript
+[human-in-the-loop-tasks]:
https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/language-sdks/typescript.html#human-in-the-loop-tasks
+[writing-tasks]:
https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/language-sdks/typescript.html#writing-tasks
+[building-a-bundle]:
https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/language-sdks/typescript.html#building-a-bundle
+[deploying]:
https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/language-sdks/typescript.html#deploying
+[limitations]:
https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/language-sdks/typescript.html#limitations
## Compatibility matrix
diff --git a/ts-sdk/docs/index.md b/ts-sdk/docs/index.md
index c3ba05b440d..2f46c154cf0 100644
--- a/ts-sdk/docs/index.md
+++ b/ts-sdk/docs/index.md
@@ -19,59 +19,56 @@
# Apache Airflow TypeScript SDK
-Write Apache Airflow Dags and tasks in TypeScript. Declare a whole Dag in
TypeScript, or implement the stub
-tasks of a Python Dag with `TaskHandler`.
+Write Apache Airflow Dags in TypeScript, and run them on Node.js.
> **Note**
-> This package is **0.1.0-beta1**: the API may change, it requires **Node
22+**, and
+> This package is **1.0.0-beta1**: the API may change, it requires **Node
22+**, and
> it is **ESM-only**.
-## Getting Started
+## Quick Start
Install the beta package from npm, and `esbuild` to build the bundle:
```bash
-npm install [email protected]
+npm install [email protected]
npm install --save-dev esbuild
```
-Declare a Dag with `Dag`, and its tasks with `dag.task`. Calling a task names
its inputs, which is how the
-tasks depend on each other:
+Declare a Dag in `src/main.ts`. Calling a task names its inputs, which is how
the tasks depend on each other:
```ts
import { Bundle, Dag } from "apache-airflow-ts-sdk";
-const dag = new Dag("ts_etl", { schedule: "@daily", queue: "typescript" });
+const dag = new Dag("ts_hello", { schedule: "@daily", queue: "typescript" });
const extract = dag.task("extract", async (): Promise<number> => 42);
-const load = dag.task("load", async ({ rows }: { rows: number }) => {
- console.log(`loading ${rows} rows`);
+const report = dag.task("report", async ({ rows }: { rows: number }) => {
+ console.log(`extracted ${rows} rows`);
});
-load({ rows: extract() });
+report({ rows: extract() });
await new Bundle(dag).serve();
```
-To implement a stub task of a Python Dag, bind a function to its `dag_id` and
`task_id` with `TaskHandler`:
+Build it into a single file:
-```ts
-import { Bundle, getClient, getContext, TaskHandler } from
"apache-airflow-ts-sdk";
+```bash
+npx airflow-ts-pack src/main.ts --outdir dist
+```
-export async function sayHello() {
- const greeting = await getClient().getVariable("greeting");
- return { message: `Hello from ${getContext().taskId}: ${greeting}` };
-}
+Put `dist/bundle.min.mjs` in a Dag bundle, such as the default `dags-folder`
bundle, and send the `typescript`
+queue to the Node.js coordinator in `airflow.cfg`:
-await new Bundle(new TaskHandler("example_dag", "say_hello",
sayHello)).serve();
+```ini
+[sdk]
+coordinators = {"ts": {"classpath":
"airflow.sdk.coordinators.node.NodeCoordinator"}}
+queue_to_coordinator = {"typescript": "ts"}
```
-In both cases a handler is a plain function. `getContext()` returns the
`TaskContext` and `getClient()` the
-`TaskClient` while it runs, and a value it returns becomes the task's
`return_value` XCom.
-
See the
[TypeScript SDK
guide](https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/language-sdks/typescript.html)
-for building and deploying a bundle, and for everything a Dag declared in
TypeScript can do.
+for everything a Dag declared in TypeScript can do, and for building and
deploying a bundle.
## API Reference
diff --git a/ts-sdk/docs/package-lock.json b/ts-sdk/docs/package-lock.json
index d3b69dabace..233fedc7a06 100644
--- a/ts-sdk/docs/package-lock.json
+++ b/ts-sdk/docs/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "ts-sdk-docs-toolchain",
- "version": "0.1.0-beta1",
+ "version": "1.0.0-beta1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "ts-sdk-docs-toolchain",
- "version": "0.1.0-beta1",
+ "version": "1.0.0-beta1",
"devDependencies": {
"@clean-jsdoc-theme/typedoc": "^5.1.1",
"@msgpack/msgpack": "^3.1.2",
diff --git a/ts-sdk/docs/package.json b/ts-sdk/docs/package.json
index a0579cca8c0..ba4e94590e4 100644
--- a/ts-sdk/docs/package.json
+++ b/ts-sdk/docs/package.json
@@ -1,6 +1,6 @@
{
"name": "ts-sdk-docs-toolchain",
- "version": "0.1.0-beta1",
+ "version": "1.0.0-beta1",
"private": true,
"description": "Documentation toolchain for the Apache Airflow TypeScript
SDK (ts-sdk). TypeDoc renders the API reference straight to HTML with the
clean-jsdoc-theme; it is kept in its own package so the docs-only dependencies
stay out of the SDK's runtime and build toolchain.",
"type": "module",
diff --git a/ts-sdk/docs/tests/deployment-checks.test.mjs
b/ts-sdk/docs/tests/deployment-checks.test.mjs
index df4bc9acd9d..e2f08a53d76 100644
--- a/ts-sdk/docs/tests/deployment-checks.test.mjs
+++ b/ts-sdk/docs/tests/deployment-checks.test.mjs
@@ -35,8 +35,8 @@ const FONT_TAGS =
'<link rel="preconnect" href="https://fonts.gstatic.com"; crossorigin />' +
'<link rel="stylesheet"
href="https://fonts.googleapis.com/css2?family=Source+Serif+4:wght@400;500;600;700&family=Roboto:wght@400;500;600;700&display=swap";
/>';
-const BASE = "/docs/ts-sdk/0.1.0-beta1";
-const VERSION = "0.1.0-beta1";
+const BASE = "/docs/ts-sdk/1.0.0-beta1";
+const VERSION = "1.0.0-beta1";
describe("normalizeBasePath", () => {
it("strips trailing slashes", () => {
diff --git a/ts-sdk/example/README.md b/ts-sdk/example/README.md
index 3e189a7eb21..7e5a40f0105 100644
--- a/ts-sdk/example/README.md
+++ b/ts-sdk/example/README.md
@@ -17,50 +17,31 @@
under the License.
-->
-# TypeScript Coordinator Runtime Example
+# TypeScript SDK examples
-This example shows both ways to use the TypeScript SDK, served from one bundle:
+Example Dags for the Apache Airflow TypeScript SDK. See the [TypeScript SDK
guide][guide] for the full documentation.
-- `src/native.ts` declares the `typescript_native_example` Dag entirely in
TypeScript: its schedule, task
- options and dependencies, including a task group, a condition, a multi-way
branch and a triggered Dag run.
-- `dags/typescript_example.py` and `dags/typescript_taskflow_example.py`
declare two Python Dags with stub
- tasks, and `src/main.ts` and `src/taskflow.ts` implement them with a
`TaskHandler` per stub task.
- Both Dags have a task called `build_message`; a handler names its `(dag_id,
task_id)` pair, so the two are
- different tasks with different bodies.
-- `dist/bundle.min.mjs` is the built bundle Airflow runs.
-
-The build uses the SDK's `airflow-ts-pack` tool, which builds the entry module
and everything it imports into
-that single file.
+[guide]:
https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/language-sdks/typescript.html
## Build
-Build the SDK first so the example can import the local package:
+Build the SDK, then the example bundles into `dist/`:
```bash
cd ts-sdk
pnpm install
pnpm run build
-```
-
-Build the example bundle and its metadata:
-```bash
-cd ts-sdk/example
+cd example
pnpm install
-pnpm run build
+pnpm run build # dist/bundle.min.mjs
+pnpm run build:hitl # dist/hitl.min.mjs
+pnpm run build:ai-approval # dist/ai-approval.min.mjs
```
-The coordinator expects this layout:
+## Configure Airflow
-```text
-ts-sdk/example/dist/
- bundle.min.mjs
-```
-
-## Airflow Configuration
-
-Register the example bundle directory as a Dag bundle, and configure Airflow to
-route the `typescript` queue to the Node coordinator that reads it:
+Register `dist/` as a Dag bundle, and route the `typescript` queue to the
Node.js coordinator:
```bash
export AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST='[
@@ -80,13 +61,38 @@ export AIRFLOW__SDK__COORDINATORS='{
export AIRFLOW__SDK__QUEUE_TO_COORDINATOR='{"typescript": "ts"}'
```
-The Dag processor parses `typescript_native_example` from `bundle.min.mjs`
with `node`, so it needs
-`node` on `PATH`, or set `node_executable` in the coordinator's `kwargs`. See
-[Parsing native
Dags](../../airflow-core/docs/authoring-and-scheduling/language-sdks/typescript.rst#parsing-native-dags).
+The workers and the Dag processor need Node.js 22 or later on `PATH`.
+
+## Dag definition
+
+These Dags are declared entirely in TypeScript:
+
+| Dag | Source | Shows
|
+| --------------------------- | -------------------- |
--------------------------------------------------------------------------------------
|
+| `typescript_native_example` | `src/native.ts` | Task inputs, a task
group, `dag.if`, `dag.switch`, and `triggerDagRun` |
+| `ts_hitl_*` | `src/hitl.ts` | One Dag per
human-in-the-loop feature, such as approvals, options, timeouts, and forms |
+| `ts_hitl_ai_approval` | `src/ai-approval.ts` | An agent that asks a
person to review its tool call before it acts |
+
+`typescript_native_example` triggers `typescript_example`, so set up the
[Python Dags](#python-dag-with-stub-taskhandler) too.
+Set the `typescript_native_cadence` Variable to `weekly` to take the other
branch of `dag.switch`.
+
+```bash
+airflow dags trigger typescript_native_example
+```
+
+Answer the human-in-the-loop requests on the **Required Actions** page.
-Copy both files in `dags/` into your Airflow Dags folder.
+## Python Dag with stub TaskHandler
-The example also reads one Variable and one Connection:
+These Dags are declared in Python, and their stub tasks are implemented in
TypeScript:
+
+| Dag | Source
| Shows |
+| ----------------------------- |
-------------------------------------------------------- |
------------------------------------------------------------ |
+| `typescript_example` | `dags/typescript_example.py`, `src/main.ts`
| Variables, Connections, XComs, and the task state store |
+| `typescript_taskflow_example` | `dags/typescript_taskflow_example.py`,
`src/taskflow.ts` | TaskFlow arguments passed from Python to TypeScript
handlers |
+
+Copy both files in `dags/` into your Airflow Dags folder, and create the
Variable and Connection
+`typescript_example` reads:
```bash
airflow variables set typescript_example_greeting "hello from Airflow"
@@ -97,13 +103,7 @@ airflow connections add typescript_example_http \
--conn-password pass
```
-`write_and_delete_variable` writes the Variables it needs: it records the run
id in
-`typescript_example_last_run` and deletes the `typescript_example_scratch`
Variable it has just written.
-
-Then start Airflow and trigger the Dag:
-
```bash
airflow dags trigger typescript_example
airflow dags trigger typescript_taskflow_example
-airflow dags trigger typescript_native_example
```
diff --git a/ts-sdk/example/package.json b/ts-sdk/example/package.json
index 3a865750a6c..d5e10ab3f57 100644
--- a/ts-sdk/example/package.json
+++ b/ts-sdk/example/package.json
@@ -1,7 +1,7 @@
{
"name": "apache-airflow-ts-sdk-example",
"private": true,
- "version": "0.1.0-beta1",
+ "version": "1.0.0-beta1",
"type": "module",
"license": "Apache-2.0",
"scripts": {
diff --git a/ts-sdk/package.json b/ts-sdk/package.json
index 61aaafd6264..f6c1dbba742 100644
--- a/ts-sdk/package.json
+++ b/ts-sdk/package.json
@@ -1,6 +1,6 @@
{
"name": "apache-airflow-ts-sdk",
- "version": "0.1.0-beta1",
+ "version": "1.0.0-beta1",
"packageManager": "[email protected]",
"description": "TypeScript Task SDK for Apache Airflow task handlers",
"license": "Apache-2.0",
