ashb commented on code in PR #72043: URL: https://github.com/apache/airflow/pull/72043#discussion_r4024908181
########## go-sdk/adr/0006-mixed-lang-task-handler-interface.md: ########## @@ -0,0 +1,169 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + --> + +# 6. Bundle registration and Mixed Lang task handlers + +Date: 2026-09-09 + +## Status + +Proposed. + +## Decision + +1. **A "bundle" is a value the author builds.** `airflow.Bundle()` returns a `*airflow.BundleRef`; + `main` reads build, register, serve, with `bundle.Serve()` as its last statement. + It replaces `BundleProvider` and `Registry`, the callback and the write half of the same bundle. +2. **`bundle.Register(items ...airflow.Registraterable)`** is the single registration verb, taking native Dags and task handlers. +3. **A Go bundle registers task handlers, not Dags**: `airflow.TaskHandler(dagId, taskId, fn)`, the Go body for a task Python declares with `@task.stub`. +4. **Both dag_id and task_id are written out on TaskHandler definition**, because Python owns them; nothing is derived from the Go function name. +5. **Every handler takes an `airflow.Context` first**: a struct embedding `context.Context`, exposing `Logger()`, `Client()`, `TaskInstance()`, and `DagRun()`. + What Airflow supplies a task arrives as a method on that value rather than as a parameter of its own. +6. **Every remaining parameter is data**, bound positionally, or by field when it is a single struct: `arg:"..."` when tagged, else the folded Go field name. + +## Context + +Python owns everything but the body of a Mixed Lang task: `@task.stub` declares the task, its arguments, and its place in the graph. +The Go side has no Dag to define, so Dag vocabulary misleads. + +Renaming the Go function must not change which task body Airflow matches, so `TaskHandler` names the dag_id and the task_id explicitly instead of inferring them from the Go function name. +Additionally, the TaskHandler shouldn't accept any spec as it should only define the implementation of stub operator, so the `airflow.TaskHandler(dag_id, task_id, fn)` is a much cleaner interface. + +Registration is inverted today. An author declares a struct with no state, asserts it implements +`v1.BundleProvider`, fills in `RegisterDags(dagbag v1.Registry) error`, and hands the struct to +`bundlev1server.Serve` — three concepts and an empty type before a single task is declared. + +The term naming should be refined to reduce the new terminologies across user interface. +The `Registry` should be `Bundle` and the `AddDag` is mis-used for registering the TaskHandler. + +The shipped signature (#70209) injects `sdk.TIRunContext`, `*slog.Logger`, and `sdk.Client` by type. +Calling them still needs the `context.Context` passed in by hand, which is awkward from a Go author's perspective. +Exposing the logger and the client on the context itself removes that, and `airflow.Context` in the Signature section below is that shape. Review Comment: How does that remove the need to pass in the context? The only way I can see we would do that is by embedding the context in the struct we return, and that is exactly the thing that the Go stdlib docs warn against isn't it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
