TimurRakhmatullin86 opened a new pull request, #69665:
URL: https://github.com/apache/airflow/pull/69665
When `AwaitMessageTrigger` is used without an `apply_function` (supported
since #55437), the trigger calls `message.value().decode("utf-8")` on every
polled message. For a tombstone message — `value()` is `None`, a routine
occurrence on log-compacted topics where a null value is the deletion marker —
this raises `AttributeError: 'NoneType' object has no attribute 'decode'`,
crashing the trigger and failing the deferred task.
This PR treats a tombstone the same way as any other non-matching message:
no event is emitted, the offset is committed (when `commit_offset=True`,
matching the existing non-matching path so the tombstone is not re-read forever
after a restart), and the trigger keeps polling.
Why skip rather than emit: the trigger's contract is already
truthiness-based (`if event:`), so a `None` payload cannot be yielded on this
path today, and a `TriggerEvent(None)` would carry no usable payload
downstream. Users who need to react to tombstones can already do so via
`apply_function`, which receives the raw message (including `value() is None`)
and can map it to any truthy payload. Surfacing tombstones from the
no-`apply_function` convenience path would be a separate opt-in feature;
crashing on them is a bug.
Tests:
* `test_trigger_run_tombstone_message_keeps_polling` — regression test:
`poll()` returns a null-value message; asserts the trigger neither raises nor
yields and commits the tombstone offset. Fails on `main` with the exact
`AttributeError` above.
* `test_trigger_run_without_apply_function_yields_message_value` — asserts
the happy path still yields the decoded payload.
* `MockedMessage` now has a `value()` method: the `apply_function=None` case
of `test_trigger_run_good` previously passed for the wrong reason — the mock
lacked `value()`, so the task finished via this same `AttributeError`,
satisfying `task.done() is True`.
No behavior change for non-null messages or when `apply_function` is set.
Empty (`b""`) payloads were already treated as non-events by the `if event:`
check — unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
