seanghaeli opened a new pull request, #74779:
URL: https://github.com/apache/airflow/pull/74779
Deadline callbacks run by an executor can read Connections and Variables
through the callback supervisor, but not XComs. That means an alert can't
include an upstream task's output. This PR serves `GetXCom` in the callback
supervisor with the shared `handle_get_xcom` handler. It also adds
`context["xcom_pull"]` to the callback context, bound to the callback's Dag
run, so a callback only has to name the task, like
`context["xcom_pull"]("extract")`. Callbacks have no task instance, so this
helper takes the place of `ti.xcom_pull` and users don't have to pass `dag_id`
and `run_id` themselves.
Only reads are exposed. The request is served with the execution token the
supervisor gets from the callback token exchange added in #71192, so no API
routes or token scopes change. This supersedes #66611. The token-scope concern
#66611 was parked over is covered by #71192, which merged on 2026-08-29.
## E2E in Breeze
Setup: scheduler with LocalExecutor, dag-processor, triggerer and api-server
on postgres. The Dag has a `produce` task that returns `{"rows_loaded": 42,
"table": "orders"}`, a mapped `produce_mapped` task over `[10, 20, 30]` that
doubles its input, then a slow task. A `DAGRUN_QUEUED_AT + 20s` deadline
`SyncCallback` calls `context["xcom_pull"]("produce")` and
`context["xcom_pull"]("produce_mapped", map_indexes=1)`.
| | Callback state | Callback log |
|---|---|---|
| Without this change | `failed` | `Callback execution failed: KeyError:
'xcom_pull'` |
| With this change | `success` | `CALLBACK_XCOM_VALUE {'table': 'orders',
'rows_loaded': 42}`, `CALLBACK_XCOM_MAPPED 40` |
<details><summary>Raw output with this change</summary>
```
=== callback table ===
01a13d63-2c91-77ec-9314-7ed1ee076e14 | executor | success |
=== callback log ===
{"level":"warning","event":"CALLBACK_XCOM_VALUE {'table': 'orders',
'rows_loaded': 42}","filename":"e2e_callback_xcom.py","lineno":24}
{"level":"warning","event":"CALLBACK_XCOM_MAPPED
40","filename":"e2e_callback_xcom.py","lineno":30}
{"level":"info","event":"Callback executed
successfully","logger":"callback_runner","filename":"callback_supervisor.py"}
=== api-server ===
method=PATCH
path=/execution/callbacks/01a13d63-2c91-77ec-9314-7ed1ee076e14/run
status_code=204
method=GET
path=/execution/xcoms/e2e_callback_xcom/manual__2026-10-15T02:27:50.038357+00:00/produce/return_value
status_code=200
method=GET
path=/execution/xcoms/e2e_callback_xcom/manual__2026-10-15T02:27:50.038357+00:00/produce_mapped/return_value
query='map_index=1' status_code=200
```
</details>
<details><summary>Raw output without this change</summary>
```
=== callback table ===
01a13d64-dddd-7881-a662-b11310bcba5c | executor | failed | Execution failed
=== callback log ===
{"level":"error","event":"Callback failed","error":"Callback execution
failed: KeyError:
'xcom_pull'","logger":"callback_runner","filename":"callback_supervisor.py"}
{"level":"warning","event":"Process exited
abnormally","exit_code":1,"logger":"callback"}
```
</details>
The new unit tests in `test_callback_supervisor.py` pass with this change.
Without it, 4 of them fail (the two `GetXCom` request cases and the two
`xcom_pull` context cases).
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (claude-fable-5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
