[ 
https://issues.apache.org/jira/browse/AIRFLOW-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430253#comment-16430253
 ] 

ASF subversion and git services commented on AIRFLOW-2027:
----------------------------------------------------------

Commit 3c4f1fd9e668a71531ec07d1e7bd60523cb31d32 in incubator-airflow's branch 
refs/heads/master from [~aoen]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=3c4f1fd ]

[AIRFLOW-2027] Only trigger sleep in scheduler after all files have parsed

Closes #2986 from aoen/ddavydov--open_source_disab
le_unecessary_sleep_in_scheduler_loop


> Only trigger sleep in scheduler after all files have parsed
> -----------------------------------------------------------
>
>                 Key: AIRFLOW-2027
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2027
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: scheduler
>            Reporter: Dan Davydov
>            Assignee: Dan Davydov
>            Priority: Major
>             Fix For: 1.10.0
>
>
> The scheduler loop sleeps for 1 second every loop unnecessarily. Remove this 
> sleep to slightly speed up scheduling, and instead do it once all files have 
> been parsed. It can add up since it runs to every scheduler loop which runs # 
> of dags to parse/scheduler parallelism times.
> Also remove the unnecessary increased file processing interval in tests which 
> slows them down.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to