Arunodoy18 opened a new pull request, #60013: URL: https://github.com/apache/airflow/pull/60013
Fix DAGs randomly disappearing due to stale DAG detection issues Summary Fixes multiple edge cases in stale DAG detection that cause DAGs to be incorrectly marked as stale/inactive, particularly in Kubernetes deployments with transient filesystem issues. Three Root Causes Fixed: 1. Empty file list during bundle refresh When bundle directories are temporarily unavailable (NFS delays, pod restarts, volume mount issues), [list_py_file_paths()](vscode-file://vscode-app/c:/Users/aruno/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) returns empty list → ALL DAGs marked stale 2. DAGs with None relative_fileloc [None not in rel_filelocs](vscode-file://vscode-app/c:/Users/aruno/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) = True → DAG incorrectly marked stale 3. DAGs with None last_parsed_time [None + timedelta(...)](vscode-file://vscode-app/c:/Users/aruno/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) → [TypeError](vscode-file://vscode-app/c:/Users/aruno/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) (and potential stale marking) Changes: 16 lines in [manager.py](vscode-file://vscode-app/c:/Users/aruno/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) - Added guards for empty file list, None checks 3 lines in [dag.py](vscode-file://vscode-app/c:/Users/aruno/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) - Added None check 164 lines in tests - 4 comprehensive unit tests Impact: ✅ Prevents false positives ✅ Legitimate stale DAGs still detected correctly ✅ No config changes needed ✅ Works with existing STALE_DAG_THRESHOLD Branch: fix/stale-dag-detection-safeguards Closes:58717 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
