potiuk commented on code in PR #37066:
URL: https://github.com/apache/airflow/pull/37066#discussion_r1479017718
##########
airflow/providers/celery/provider.yaml:
##########
@@ -276,6 +276,21 @@ config:
type: float
example: ~
default: "1.0"
+ task_acks_late:
+ description: |
+ If an Airflow task's execution time exceeds the visibility_timeout,
Celery will re-assign the
+ task to a Celery worker, even if the original task is still running
successfully. The new task
+ instance then runs concurrently with the original task and the
Airflow UI and logs only show an
+ error message:
+ 'Task Instance Not Running' FAILED: Task is in the running state'
+ Setting task_acks_late to True will force Celery to wait until a
task is finished before a
+ new task instance is assigned. This effectively overrides the
visibility timeout.
+ See also:
+
https://docs.celeryq.dev/en/stable/reference/celery.app.task.html#celery.app.task.Task.acks_late
+ version_added: ~
Review Comment:
```suggestion
version_added: 3.6.0
```
##########
airflow/providers/celery/provider.yaml:
##########
@@ -276,6 +276,21 @@ config:
type: float
example: ~
default: "1.0"
+ task_acks_late:
+ description: |
+ If an Airflow task's execution time exceeds the visibility_timeout,
Celery will re-assign the
+ task to a Celery worker, even if the original task is still running
successfully. The new task
+ instance then runs concurrently with the original task and the
Airflow UI and logs only show an
+ error message:
+ 'Task Instance Not Running' FAILED: Task is in the running state'
+ Setting task_acks_late to True will force Celery to wait until a
task is finished before a
+ new task instance is assigned. This effectively overrides the
visibility timeout.
+ See also:
+
https://docs.celeryq.dev/en/stable/reference/celery.app.task.html#celery.app.task.Task.acks_late
+ version_added: ~
Review Comment:
@hussein-awala ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
