[ https://issues.apache.org/jira/browse/AIRFLOW-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950888#comment-16950888 ]
Ash Berlin-Taylor commented on AIRFLOW-5639: -------------------------------------------- As you say, right now we need to do this, but we are working towards making this not needed. > DagFileProcessor: parse dag files every time, which consume lots of resources > and is unnecessary > ------------------------------------------------------------------------------------------------ > > Key: AIRFLOW-5639 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5639 > Project: Apache Airflow > Issue Type: Improvement > Components: scheduler > Affects Versions: 1.10.5 > Reporter: chen xianxin > Priority: Major > > > h2. Code > [https://github.com/apache/airflow/blob/v1-10-stable/airflow/models/dagbag.py#L166-L170] > h2. Problem description > *self.file_last_changed* doesn't work, beacause it's set to empty when Dagbag > is initialized. so even no changes were made to the file, it will also be > imported. I have sure this problem by print log. > Generate bagdags from files take about 50% time of dag files processing, if > only generate bagdags when files are changed, lots of resources will be > saved. -- This message was sent by Atlassian Jira (v8.3.4#803005)