Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-26 Thread via GitHub
tchakib commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1580580045 ## airflow/config_templates/config.yml: ## @@ -500,6 +500,14 @@ core: type: integer example: ~ default: "4096" +enable_purging_stale_dags:

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-26 Thread via GitHub
uranusjr commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1580528227 ## docs/apache-airflow/core-concepts/dags.rst: ## @@ -924,3 +924,10 @@ if it fails for ``N`` number of times consecutively. we can also provide and override these

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-26 Thread via GitHub
uranusjr commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1580525790 ## airflow/config_templates/config.yml: ## @@ -500,6 +500,14 @@ core: type: integer example: ~ default: "4096" +enable_purging_stale_dags:

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-26 Thread via GitHub
uranusjr commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1580525996 ## airflow/config_templates/config.yml: ## @@ -500,6 +500,14 @@ core: type: integer example: ~ default: "4096" +enable_purging_stale_dags:

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-25 Thread via GitHub
uranusjr commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1579059967 ## airflow/dag_processing/manager.py: ## @@ -599,7 +599,10 @@ def _run_parsing_loop(self): if self.standalone_dag_processor:

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-25 Thread via GitHub
tchakib commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1579051279 ## airflow/dag_processing/manager.py: ## @@ -599,7 +599,10 @@ def _run_parsing_loop(self): if self.standalone_dag_processor:

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-25 Thread via GitHub
tchakib commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1579012744 ## airflow/dag_processing/manager.py: ## @@ -599,7 +599,10 @@ def _run_parsing_loop(self): if self.standalone_dag_processor:

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-24 Thread via GitHub
uranusjr commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1578810984 ## docs/apache-airflow/core-concepts/dags.rst: ## @@ -924,3 +924,10 @@ if it fails for ``N`` number of times consecutively. we can also provide and override these

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-24 Thread via GitHub
uranusjr commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1578810901 ## airflow/dag_processing/manager.py: ## @@ -599,7 +599,10 @@ def _run_parsing_loop(self): if self.standalone_dag_processor:

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-23 Thread via GitHub
tchakib commented on PR #39159: URL: https://github.com/apache/airflow/pull/39159#issuecomment-2071804966 > There are a couple of exceptions, but I think the configs are generally positive, i.e. this should use `scan_stale_dags` (or `enable_...`) and default to True. in fact, if it

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-23 Thread via GitHub
tchakib commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1575909062 ## docs/apache-airflow/core-concepts/dags.rst: ## @@ -924,3 +924,10 @@ if it fails for ``N`` number of times consecutively. we can also provide and override these

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-23 Thread via GitHub
tchakib commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1575908534 ## airflow/config_templates/config.yml: ## @@ -500,6 +500,13 @@ core: type: integer example: ~ default: "4096" +disalble_scan_stale_dags: +

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-23 Thread via GitHub
uranusjr commented on PR #39159: URL: https://github.com/apache/airflow/pull/39159#issuecomment-2071589726 There are a couple of exceptions, but I think the configs are generally positive, i.e. this should use `scan_stale_dags` (or `enable_...`) and default to True. -- This is an

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-22 Thread via GitHub
amoghrajesh commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1575631616 ## airflow/config_templates/config.yml: ## @@ -500,6 +500,13 @@ core: type: integer example: ~ default: "4096" +

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-22 Thread via GitHub
tchakib commented on PR #39159: URL: https://github.com/apache/airflow/pull/39159#issuecomment-2069642961 > modèles > Pourriez-vous également ajouter des cas de test pour celui-ci dans le https://github.com/apache/airflow/blob/main/tests/dag_processing/test_processor.py

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-22 Thread via GitHub
Taragolis commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1574840593 ## airflow/dag_processing/manager.py: ## @@ -599,7 +599,10 @@ def _run_parsing_loop(self): if self.standalone_dag_processor:

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-22 Thread via GitHub
Taragolis commented on code in PR #39159: URL: https://github.com/apache/airflow/pull/39159#discussion_r1574837102 ## airflow/config_templates/config.yml: ## @@ -500,6 +500,13 @@ core: type: integer example: ~ default: "4096" +disalble_scan_stale_dags:

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-22 Thread via GitHub
potiuk commented on PR #39159: URL: https://github.com/apache/airflow/pull/39159#issuecomment-2068891548 In order for that change would be considered for merging - it needs to have more explanation, documentation, configuration parameter added and described in the templates and unit

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-04-22 Thread via GitHub
boring-cyborg[bot] commented on PR #39159: URL: https://github.com/apache/airflow/pull/39159#issuecomment-2068855641 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our