hkc-8010 opened a new issue, #64890:
URL: https://github.com/apache/airflow/issues/64890
### Description
In Airflow 2, a common notifier/callback pattern was:
1. Post an alert to an external system such as Slack.
2. Persist a backlink or correlation identifier on the `DagRun` note.
3. Reuse that stored link later to update the original alert if the DagRun
is cleared and eventually succeeds.
That migration path is not straightforward in Airflow 3.
In Airflow 3 callback/notifier code, `context["dag_run"]` is no longer the
old SQLAlchemy `DagRun` contract that many Airflow 2 implementations relied on,
and the Task SDK client does not appear to expose a supported helper for
updating `DagRun.note` from callback/notifier runtime code.
There is already public API support for DagRun note patching, so this
request is specifically about having a supported callback-safe/runtime-safe way
to do that from Airflow 3 notifier code.
### Use case/motivation
A concrete use case is a DAG-level failure notifier that:
1. Fires once when the DagRun fails.
2. Collects failed task IDs for that run.
3. Posts a Slack message.
4. Stores the Slack thread URL in the DagRun note for traceability.
5. On a later callback (for example after the DagRun is cleared and
succeeds), reads that stored link and updates the original Slack message
instead of sending a brand new one.
The "list failed tasks" part is portable in Airflow 3 via
`context["task_instance"].get_task_states(...)` in a real callback execution
path.
The missing piece is the note update. In Airflow 2 this was commonly done
with ORM/session access such as `DagRunNote`, `dag_run.dag_run_note`, and
`provide_session`. In Airflow 3 that pattern is not portable, but I could not
find a supported replacement in notifier/callback context.
### Related issues
- #51816 is related only in the sense that local `dag.test()` / `airflow
dags test` debugging can be misleading for callback runtime APIs. It does not
solve the DagRun note update gap described here.
### Requested outcome
One of the following would unblock Airflow 2 -> 3 notifier migrations
cleanly:
1. Add a supported Task SDK / callback-context helper to update
`DagRun.note` from notifier code.
2. Expose a documented runtime client operation for patching the current
DagRun note from callback code.
3. If this is intentionally unsupported, document the recommended
replacement pattern for storing callback correlation metadata on a DagRun in
Airflow 3.
### Environment where this came up
- Apache Airflow 3.1.8
- DAG-level `on_failure_callback`
- Local Astro / Astronomer runtime during migration testing
### Minimal callback shape
```python
from airflow.notifications.basenotifier import BaseNotifier
class ExampleNotifier(BaseNotifier):
def notify(self, context):
dag_run = context["dag_run"]
ti = context["task_instance"]
task_states = ti.get_task_states(
dag_id=dag_run.dag_id,
run_ids=[dag_run.run_id],
)
failed_tasks = [
task_id
for task_id, state in task_states.get(dag_run.run_id, {}).items()
if state in {"failed", "upstream_failed"}
]
# Supported in Airflow 3: derive failed task list.
print(failed_tasks)
# Missing piece: supported way to persist a backlink such as a Slack
URL
# onto the DagRun note from this notifier/callback context.
```
### Why I think this belongs in OSS Airflow
This appears to be a migration and API-surface gap rather than a
managed-service issue:
- Airflow 3 callback context behavior changed relative to many Airflow 2
implementations.
- Public DagRun note patching exists, so the underlying capability is
present.
- The missing piece is a supported, documented path for callback/notifier
code to update the current DagRun note.
Thanks for taking a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
