shivaam commented on issue #69288:
URL: https://github.com/apache/airflow/issues/69288#issuecomment-5087648359
For task outputs, I think we can start with strong TypeScript-to-TypeScript
type safety.
Each task call could return a `TaskOutput<T>`. TypeScript can then check
both the task implementation's return type and whether that output is
compatible with the downstream task's input.
```ts
type Extracted = {
rows: string[];
};
type Transformed = {
count: number;
};
const extract = pipeline.task({
taskId: "extract",
run: async (): Promise<Extracted> => {
return { rows: ["a", "b"] };
},
});
const transform = pipeline.task({
taskId: "transform",
run: async ({ extracted }: { extracted: Extracted }): Promise<Transformed>
=> {
return { count: extracted.rows.length };
},
});
const extracted = extract();
const transformed = transform({ extracted });
```
Here the compiler checks that `extract` really returns `Extracted` and that
its `TaskOutput<Extracted>` can feed the `extracted` input of `transform`. The
task calls still only construct the Dag and its XCom bindings; they do not
execute the handlers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
