anishgirianish commented on PR #60855:
URL: https://github.com/apache/airflow/pull/60855#issuecomment-3902784673
@ashb Thanks for the review, I've pushed updates addressing all your
feedback. Quick summary:
Why the "ignore" behaviour: In Airflow 3.x, the scheduler now processes
executor events for RUNNING tasks (not just QUEUED as in 2.x). When Celery
redelivers a message for an already-running task, the worker hits a 409 from
the API server. Without this fix, Celery records it as a FAILURE event, and the
scheduler marks the still-running task as failed. The fix lets the supervisor
detect the 409 (task already running) and exit gracefully instead of
propagating a failure.
Changes made:
- Moved 409 handling into TaskInstanceOperations.start() (no more fragile
dict parsing)
- Replaced all broker/Celery-specific language with executor-agnostic terms
Ready for another look when you have a chance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
