potiuk opened a new pull request, #74830:
URL: https://github.com/apache/airflow/pull/74830
Deadline Alert callback `kwargs` were decoded in the scheduler and the API
server whenever a
serialized Dag's deadline alerts were loaded. Neither component calls the
callback: the decoded
callback is only used to pick the callback model by type and to copy
`serialize()` back into
`Callback.data`, which is an identity round-trip. Decoding there means
constructing user objects
in components that never use them, and it is also lossy for sync callbacks,
whose kwargs went
through `BaseSerialization` and a JSON workload before reaching the worker.
This keeps the kwargs in their serde-encoded form on the server and decodes
them at the point
of use, on both consumption paths:
- **Server:** `_decode_deadline_callback` keeps `kwargs` encoded (only a
legacy
`{"__type": "dict", "__var": ...}` container is unwrapped). Re-encoding a
decoded
`SerializedDeadlineAlert` writes them back unchanged instead of
serializing them again.
- **AsyncCallback (triggerer):** `TriggererCallback.queue` stores the
encoded kwargs on the
trigger row as-is via a new `Trigger.from_serialized_kwargs`, giving the
same stored form as
before. The triggerer decodes them with its existing pipeline when it
creates the
`CallbackTrigger`, so the triggerer itself is unchanged.
- **SyncCallback (worker):** `execute_callback` decodes the kwargs in the
callback subprocess
right before calling the user callable. The scheduler-provided `context`
is passed through.
- **SyncCallback version note:** the executor callback data now carries
`"kwargs_encoding": "serde"`. `BaseExecutor.run_workload` passes it to
`supervise_callback` only
when set, and the Task SDK decodes the kwargs only when it is present, so
callbacks created
before upgrade (no marker, already-decoded kwargs) run unchanged. A worker
with an upgraded
`apache-airflow` and an older Task SDK fails such callbacks with an error
instead of running
them with encoded arguments. Workers should upgrade `apache-airflow` and
`apache-airflow-task-sdk` together; see the significant newsfragment.
Both the current serde encoding and the legacy `{"__type", "__var"}`
encoding are handled, and
Callback rows created before upgrade (holding already-decoded kwargs) keep
working on both paths.
Tests cover:
- kwargs naming a class are not constructed during server-side decode, and
round-trip
byte-identically;
- AsyncCallback kwargs arrive decoded in the `CallbackTrigger` built the way
the triggerer
builds it;
- SyncCallback kwargs arrive decoded in the callable after a full
store -> workload JSON -> `execute_callback` round-trip;
- each of the above for current and legacy encodings;
- with the encoding marker the worker decodes, without it the kwargs are
passed as-is, and an
unknown marker fails the callback;
- the async path is unchanged.
The test that asserted the previous decode-on-load behaviour is replaced.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Opus 5 following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
