Daniel Imberman updated AIRFLOW-6175:
    Fix Version/s: 1.10.6

> Executor changes queued tasks to scheduled without removing from task queue
> ---------------------------------------------------------------------------
>                 Key: AIRFLOW-6175
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6175
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.10.6
>            Reporter: Daniel Imberman
>            Priority: Blocker
>             Fix For: 1.10.6
> There is a bug caused by scheduler_jobs refactor which leads to task failure 
> and scheduler locking.
> Essentially when a there is an overflow of tasks going into the scheduler, 
> the tasks are set back to scheduled, but are not removed from the executor's 
> queued_tasks queue.
> This means that the executor will attempt to run tasks that are in the 
> scheduled state, but those tasks will fail dependency checks. Eventually the 
> queue is filled with scheduled tasks, and the scheduler can no longer run.

This message was sent by Atlassian Jira

Reply via email to