shahar1 opened a new pull request, #73504:
URL: https://github.com/apache/airflow/pull/73504
Supersedes #67621, which has been idle since 2026-05-27. The original work
is by
@michaelpri10 and is preserved here as the first commit, authored to him.
This PR
rebases it onto current `main` (no merge commits) and adds the follow-up
work as
separate, reviewable commits.
`PubSubPullOperator`, `PubSubPullSensor` and `PubsubPullTrigger` hardcoded
`return_immediately=True`, which relies on a Pull option Google deprecated
because
it can return zero messages while a backlog exists. This makes the option
configurable, keeps `True` as the default so existing behaviour is
preserved, and
warns that the default will flip in the first Google provider major release
after
March 31, 2027.
### Commits
| Commit | Author | What |
|---|---|---|
| `Make return_immediately configurable for the Pub/Sub modules` |
@michaelpri10 | The original feature, squashed from his 7 commits into his net
delta |
| `Warn when return_immediately is unset rather than when it is set` |
@shahar1 | Inverts the deprecation trigger; extends it to `PubsubPullTrigger`;
shares one message constant |
| `Set return_immediately in the Pub/Sub examples` | @shahar1 | Example Dags
no longer teach the deprecated pattern |
| `Document the Pub/Sub return_immediately changes for users` | @shahar1 |
Changelog note and operator guide |
### What changed relative to #67621
- **The deprecation now fires when `return_immediately` is *unset*, not when
it is
set.** Warning only on explicit use leaves everyone still on the implicit
default
hearing nothing, and nags users who already made a deliberate choice.
**This is the
main decision worth a maintainer's opinion.**
- **`PubsubPullTrigger` is covered too.** It was uncovered, yet the
`google+pubsub`
scheme constructs it directly for asset watchers, so those Dag authors
silently got
the deprecated default. Its message names the subscription, because the
trigger is
built inside `MessageQueueTrigger.serialize()` where `stacklevel=2`
resolves to
`common.messaging`'s file rather than the user's watcher. The operator and
sensor
deliberately do *not* interpolate it — `subscription` is a template field
there, so
at `__init__` time it can still hold an unrendered Jinja expression, which
`validate-operators-init` rejects.
- **Deferrable `PubSubPullSensor` now respects `return_immediately`.** It
previously
dropped the argument when handing off to the trigger, so it always behaved
as
`True`. This is a real behaviour change for anyone already using
`deferrable=True, return_immediately=False`, and it is called out in the
changelog.
- **The warning message lives in one constant** rather than three copies, so
the
removal date cannot drift. The `warnings.warn` call stays in each class:
`fixup_decorator_warning_stack` only adjusts the stack for modules that
define an
operator, so moving the call out would break the frame the warning points
at.
- **Tests**: dropped a module-level `filterwarnings` mark that silenced every
`return_immediately` deprecation in these files, and added assertions that
an unset
argument still resolves to `True` — the backward-compatibility contract of
the
deprecation, which nothing pinned.
### Verification
- Full `Providers[google]` suite: 5379 passed, 41 skipped
- `breeze run mypy`, `breeze build-docs google`, `prek` pre-commit and
manual stages: all clean
- Each commit is green on its own
- Exercised against live GCP Pub/Sub: operator with
`return_immediately=False` and a
backlog pulled messages; with `True` on an empty subscription returned
`[]` without
hanging; the sensor and async trigger both pulled; an unset construction
warned and
resolved to `True`
Provider system tests could not be run locally —
`providers/google/tests/system/google/conftest.py:21`
raises `ModuleNotFoundError: No module named 'system.openlineage'`, which is
pre-existing
on `main` and unrelated to this change.
Please use **squash-and-merge**: the branch carries older commits from the
original PR
whose subjects predate the current message conventions.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Opus 5)
Generated-by: Claude Code (Opus 5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
