Hi All,
We are observing intermittently  that Tasks get stuck in queued state and never 
get executed by Airflow.
On debugging it we found that one of the queued dependency was not met due to 
which task did not move from queued to running state. So task remained in 
queued state.
(are_dependencies_met function returned false for QUEUE_DEPS inside 
_check_and_change_state_before_execution)

By looking into scheduler code it seems that scheduler does not reschedule the 
queued state tasks due to which task never got added to executor queue again 
and remained stuck in queued state.

There is a logic inside _check_and_change_state_before_execution function to 
move the task from queued to None state(which gets picked by scheduler for 
rescheduling) if RUN_DEPS are not met but this logic seems to be missing for 
QUEUE_DEPS. 

It seems that task should be moved to None state even if QUEUE_DEPS are not met.
Any help/pointer on this would be useful.

Thanks,
Raman Gupta

 

Reply via email to