Alejandro created AIRFLOW-4553:
----------------------------------
Summary: Scheduler only schedules one task at a time with
CeleryExecutor
Key: AIRFLOW-4553
URL: https://issues.apache.org/jira/browse/AIRFLOW-4553
Project: Apache Airflow
Issue Type: Bug
Components: scheduler
Affects Versions: 1.10.3, 1.10.2
Reporter: Alejandro
I've recently upgraded from Airflow 1.9.0 to Airflow 1.10.3. There has been a
change in how the scheduler put tasks up for execution that have affected some
of our DAGs. We use the CeleryExecutor.
On Airflow 1.9.0 the scheduler would set as `scheduled` as many tasks as it
could, For example in a DAG with 5 independent tasks and enough concurrency
set, the scheduler would queue the 5 tasks for execution and all 5 tasks could
potentially start running.
On Airflow 1.10.3 the scheduler would set as `scheduled` only one task at a
time, For example in a DAG with 5 independent tasks and enough concurrency set,
the scheduler would queue the 1 task for execution and the other 4 would not
stay with a None state.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
