The GitHub Actions job "Tests" on airflow.git/cleanup_add_files_to_queue has succeeded. Run started by GitHub user jedcunningham (triggered by jedcunningham).
Head commit for run: 5020150c9a92b9e987e8a676109eb8c223769f73 / Jed Cunningham <[email protected]> Refactor DAG file queuing and fix redundant processing Renamed `add_files_to_queue` to `_add_new_files_to_queue` in `DagFileProcessorManager` to reduce confusion with `_add_files_to_queue` and better reflect its internal usage for newly discovered files. We also call the method only after a bundle has refreshed - we can't find new files without that, so doing it in every loop is wasteful. The method now checks `_processors` in addition to `_file_stats` before adding files. This prevents a race condition where files currently being processed (which don't yet have stats) were erroneously re-added to the parsing queue. The method also now results in the `dag_processing.file_path_queue_size` gauge being emitted after adding new files to the queue, and reduces log noise by having a single log line vs one per file. Report URL: https://github.com/apache/airflow/actions/runs/20719018909 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
