The GitHub Actions job "Backport Commit" on airflow.git/main has failed. Run started by GitHub user kaxil (triggered by kaxil).
Head commit for run: 57a7c64a77503fef4eb7c6801a28a628a4098535 / Nick Stenning <[email protected]> Fix broken `dag_processing.total_parse_time` metric (#62128) DagFileProcessorManager has been emitting a nonsense value for `dag_processing.total_parse_time` since 8774f28d76, which reversed the order in which `emit_metrics` and `prepare_file_queue` (then called `prepare_file_path_queue`) were called. As `prepare_file_path_queue` was responsible for resetting the value of `self._parsing_start_time`, the assumption made by `emit_metrics` was that it would be called once the file queue had been cleared, but crucially before `prepare_file_queue` was called to refill the queue. Additionally, there was no guarantee that we'd parsed any files at all since the last time the metric was emitted. If no work was due, we'd gladly emit near-zero metrics every time around the while loop. I've rearranged things in such a way that I hope will be harder to accidentally break in future: - `self._parsing_start_time` may be reset whenever files are added to the queue, if it was not set already. - metrics are emitted when `prepare_file_queue` is called -- when the queue is empty -- but only if `self._parsing_start_time` is set, meaning only if we've actually parsed any files since the last time metrics were emitted. Together, this means we should now emit metrics once per parsing loop. I've added a test which fails on main and passes on this branch. Report URL: https://github.com/apache/airflow/actions/runs/22245489670 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
