mhenc commented on code in PR #25935:
URL: https://github.com/apache/airflow/pull/25935#discussion_r960583751
##########
airflow/config_templates/config.yml:
##########
@@ -2121,6 +2121,14 @@
type: integer
example: ~
default: "20"
+ - name: stalled_dags_update_timeout
Review Comment:
There is already config `deactivate_stale_dags_interval`:
https://github.com/apache/airflow/blob/main/airflow/config_templates/default_airflow.cfg#L982
but it is used by DagProcessor and has slightly different meaning.
I believe 60seconds there may be too low - as if dag parsing process takes
more that 1 minute (many dags or high min_file_process_interval) then Scheduler
may start removing some dags (as they were not updated withing last 60 seconds).
I put 10minutes there - as this sounds like a safer value.
Other idea: instead of introducing new config we may simply use
`min_file_process_interval * X` with e.g X=10?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
