Xiaodong DENG created AIRFLOW-2922:
--------------------------------------
Summary: Potential deal-lock bug in CeleryExecutor()
Key: AIRFLOW-2922
URL: https://issues.apache.org/jira/browse/AIRFLOW-2922
Project: Apache Airflow
Issue Type: Improvement
Components: executor
Reporter: Xiaodong DENG
Assignee: Xiaodong DENG
Code:
[https://github.com/apache/incubator-airflow/blob/404be4b021e803239f48e4dd5705759cc77aeb92/airflow/executors/celery_executor.py]
Celery Task states normally change in ways like “PENDING -> STARTED ->
SUCCESS/FAILURE”
([http://docs.celeryproject.org/en/latest/reference/celery.states.html]).
In lines 107 and 108, it’s `task.state` rather than `state`, i.e. it will
reflect the latest real-time state of the Celery task.
Let’s imagine: task state becomes “STARTED” (initial state is “PENDING”), then
the if-elif-else block will be triggered. It’s possible the Celery task state
becomes “SUCCESS” when whichever line between 94-105 is running. At line 108,
the latest state of the task in _*self.last_state*_ will be changed to
“SUCCESS” rather than “STARTED” because it’s referring to `task.state` rather
than variable `state`.
Then this task will be dead-locked because the if-elif-else block will never be
triggered for it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
