The GitHub Actions job "Tests" on airflow.git/backport-62128 has failed. Run started by GitHub user vatsrahul1001 (triggered by vatsrahul1001).
Head commit for run: ec1c019e4479278ff8c18ce96b0f72aeb4946b4e / Nick Stenning <[email protected]> Fix broken `dag_processing.total_parse_time` metric (#62128) DagFileProcessorManager has been emitting a nonsense value for `dag_processing.total_parse_time` since 8774f28d76, which reversed the order in which `emit_metrics` and `prepare_file_queue` (then called `prepare_file_path_queue`) were called. As `prepare_file_path_queue` was responsible for resetting the value of `self._parsing_start_time`, the assumption made by `emit_metrics` was that it would be called once the file queue had been cleared, but crucially before `prepare_file_queue` was called to refill the queue. Additionally, there was no guarantee that we'd parsed any files at all since the last time the metric was emitted. If no work was due, we'd gladly emit near-zero metrics every time around the while loop. I've rearranged things in such a way that I hope will be harder to accidentally break in future: - `self._parsing_start_time` may be reset whenever files are added to the queue, if it was not set already. - metrics are emitted when `prepare_file_queue` is called -- when the queue is empty -- but only if `self._parsing_start_time` is set, meaning only if we've actually parsed any files since the last time metrics were emitted. Together, this means we should now emit metrics once per parsing loop. I've added a test which fails on main and passes on this branch. (cherry picked from commit 57a7c64a77503fef4eb7c6801a28a628a4098535) Report URL: https://github.com/apache/airflow/actions/runs/22611232911 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
