Xiaodong DENG created AIRFLOW-2814:
--------------------------------------
Summary: Default Arg "file_process_interval" for class
SchedulerJob is inconsistent with doc
Key: AIRFLOW-2814
URL: https://issues.apache.org/jira/browse/AIRFLOW-2814
Project: Apache Airflow
Issue Type: Bug
Components: scheduler
Reporter: Xiaodong DENG
Assignee: Xiaodong DENG
h2. Backgrond
In
[https://github.com/XD-DENG/incubator-airflow/blob/master/airflow/jobs.py#L592]
, it was mentioned the default value of argument *file_process_interval* should
be 3 minutes (*file_process_interval:* Parse and schedule each file no faster
than this interval).
The value is normally parsed from the default configuration. However, in the
default config_template, its value is 0 rather than 180 seconds
([https://github.com/XD-DENG/incubator-airflow/blob/master/airflow/config_templates/default_airflow.cfg#L432]
).
h2. Issue
This means that actually that each file is parsed and scheduled without letting
Airflow "rest". This conflicts with the design purpose (by default let it be
180 seconds) and may affect performance significantly.
h2. My Proposal
Change the value in the config template from 0 to 180.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
