[
https://issues.apache.org/jira/browse/AIRFLOW-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16341861#comment-16341861
]
Dan Davydov commented on AIRFLOW-2027:
--------------------------------------
the PR I'm about to cut will sleep 1s - time it took to parse dags (so still 1s
minimum). Not greatly concerned about the busy looping really, but the log spam
is an issue.
We have found in high load situations, 10% of the delay was caused by this
sleep.
> Remove unnecessary 1s sleep in scheduler loop
> ---------------------------------------------
>
> Key: AIRFLOW-2027
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2027
> Project: Apache Airflow
> Issue Type: Improvement
> Components: scheduler
> Reporter: Dan Davydov
> Assignee: Dan Davydov
> Priority: Major
>
> The scheduler loop sleeps for 1 second every loop unnecessarily. Remove this
> sleep to slightly speed up scheduling. It can add up since it runs to every
> scheduler loop which runs # of dags to parse/scheduler parallelism times.
>
> Also remove the unnecessary increased file processing interval in tests which
> slows them down.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
