[
https://issues.apache.org/jira/browse/AIRFLOW-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15986765#comment-15986765
]
Gerard Toonstra commented on AIRFLOW-1133:
------------------------------------------
Could be related to this issue too.
> More tasks than the concurrency limit can run
> ---------------------------------------------
>
> Key: AIRFLOW-1133
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1133
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Alex Guziel
>
> There are two ways to put checks on dag concurrency:
> 1) The scheduler not queueing too many tasks (via checking the amount of
> tasks running)
> 2) The worker checking that not too many tasks are running (via the db)
> Right now, both have issues.
> 1 doesn't considered queued tasks which may not be running now, but will be
> running soon. Hopefully, check 2 should catch it, but it does not check the
> condition properly as it only locks the row, and it seems locking the dag
> would also be expensive.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
