iralp opened a new pull request, #75373:
URL: https://github.com/apache/airflow/pull/75373
Three doc surfaces describe a broker redelivery as terminating the
still-running original task:
- the `[celery] task_acks_late` config description in `provider.yaml` (and
its
generated mirror in `get_provider_info.py`), which also describes the
failure
as surfacing only as `'Task Instance Not Running' FAILED: Task is in the
running state'`
- the runtime warning from `_broker_transport_options` when no
`visibility_timeout` is configured
- `celery_executor.rst`, which asks for a visibility timeout above the
longest
task ETA without saying what happens if it is not
That stopped being the only outcome in `apache-airflow-providers-celery`
3.18.0
(#64052): `celery_executor_utils` raises `Ignore()` on
`TaskAlreadyRunningError`, so the redelivered message is discarded and the
original task keeps running. It also needs Airflow 3.1.9 or newer, because
the
409 it relies on comes from the Execution API and the provider still gates on
`AIRFLOW_V_3_1_9_PLUS` — so the existing text remains correct below that
floor
and is now scoped to those versions rather than replaced.
Measured on Airflow 3.3.2 with provider 3.24.0: a 301-second task under a
60-second visibility timeout reached `success`, and the duplicate logged
`Task already running elsewhere, ignoring redelivered message`.
Neither surface named a version floor, and the provider doc set had a single
bullet about visibility timeout with no explanation of the mechanism, so this
adds a "Visibility timeout and long-running tasks" section covering
`task_acks_late`'s interaction with redelivery, both version floors, and
SQS's
43200-second cap.
Follow-up to #62869, which added the warning text. It was accurate for the
provider versions available at the time; this PR scopes it to those versions.
### Validation
- `providers/celery/tests/unit/celery/executors/test_celery_executor.py`: 81
passed, 18 skipped
- `get_provider_info.py` regenerated from `provider.yaml`, byte-identical to
the committed file
- `provider.yaml` passes both the provider schema and the runtime
`provider_info` schema
- ruff check/format, mypy, yamllint, codespell clean on the changed files
- new prose checked against hunspell en_US plus `docs/spelling_wordlist.txt`
Sphinx docs build not yet run locally; I am relying on the CI `build-docs`
job
for the new RST section and its cross-reference.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Opus 5 following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
I reviewed and verified the change myself: I confirmed the
`TaskAlreadyRunningError` / `Ignore()` path in `celery_executor_utils.py`,
the
`AIRFLOW_V_3_1_9_PLUS` gate it depends on, and the 3.18.0 changelog entry for
#64052, and I reproduced the behavior on a live deployment as described
above.
The final responsibility for this PR is mine.
---
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
