This is an automated email from the ASF dual-hosted git repository.
amoghdesai pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new b89044da97e Add "disappearing DAGs" section on FAQ doc (#49987)
b89044da97e is described below
commit b89044da97efba3ef43ec6c2441c7026ab61fc69
Author: rtrindvg <[email protected]>
AuthorDate: Mon Jun 2 05:07:59 2025 -0300
Add "disappearing DAGs" section on FAQ doc (#49987)
---------
Co-authored-by: Amogh Desai <[email protected]>
---
airflow-core/docs/faq.rst | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
diff --git a/airflow-core/docs/faq.rst b/airflow-core/docs/faq.rst
index 8727deac08e..9eb5fa79ad0 100644
--- a/airflow-core/docs/faq.rst
+++ b/airflow-core/docs/faq.rst
@@ -206,6 +206,37 @@ until ``min_file_process_interval`` is reached since DAG
Parser will look for mo
return dag
+What to do if you see disappearing DAGs on UI?
+----------------------------------------------
+There are several reasons why DAGs might disappear from the UI. Common causes
include:
+
+* **Total parsing of all DAGs is too long** - If parsing takes longer than
:ref:`config:core__dagbag_import_timeout`,
+ files may not be processed completely. This often occurs when DAGs don't
follow
+ :ref:`DAG writing best practices<best_practice:writing_a_dag>` like:
+
+ * Excessive top-level code execution
+ * External system calls during parsing
+ * Complex dynamic DAG generation
+
+* **Inconsistent dynamic DAG generation** - DAGs created through
+ :doc:`dynamic generation </howto/dynamic-dag-generation>` must produce
stable DAG IDs across parses.
+ Verify consistency by running ``python your_dag_file.py`` repeatedly.
+
+* **File processing configuration issues** - A certain combination of
parameters may lead to scenarios which certain DAGs are less likely to be
processed at each loop. Check these parameters:
+
+ * :ref:`config:dag_processor__file_parsing_sort_mode` - Ensure sorting
method matches your sync strategy
+ * :ref:`config:dag_processor__parsing_processes` - Number of parallel parsers
+ * :ref:`config:scheduler__parsing_cleanup_interval` - Controls stale DAG
cleanup frequency
+ * :ref:`config:scheduler__dag_stale_not_seen_duration` - Time threshold for
marking DAGs as stale
+
+* **File synchronization problems** - Common with git-sync setups:
+
+ * Symbolic link swapping delays
+ * Permission changes during sync
+ * `mtime` preservation issues
+
+* **Time synchronization issues** - Ensure all nodes (database, schedulers,
workers) use NTP with <1s clock drift.
+
DAG construction
^^^^^^^^^^^^^^^^
