SamWheating opened a new pull request #21399: URL: https://github.com/apache/airflow/pull/21399
Re: https://github.com/apache/airflow/issues/21397 By moving this logic into the `DagFileProcessorManager` and running it across all processed file periodically, we can prevent the use of un-indexed queries. The basic logic is that we can look at the last processed time of a file (for a given processor) and compare that to the `last_parsed_time` of an entry in the `dag` table. If the file has been processed significantly more recently than the DAG has been updated, then its safe to assume that the DAG is missing and can be marked inactive. ### Todo: - [ ] Improve test coverage - [ ] Exposed new tuneable parameters in the config -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
