abhijeets25012-tech opened a new pull request, #76180: URL: https://github.com/apache/airflow/pull/76180
`DagVersion.version_number` is the scheduler's definition of a Dag's current version, but several readers sorted by a timestamp or a row id instead. After an in-place bundle refresh, or when a row belonging to an older bundle version is inserted later, those readers disagreed with the scheduler about which version is current. This adds `DagVersion.latest_version_subquery()` as the single definition of "current version" and routes the time-ordered readers through it: * `db_cleanup`'s `keep_last` for `dag_version` (new `keep_last_column_name` option on `_TableConfig`, so the cleanup keeps the highest `version_number` rather than the newest row) * `SerializedDagModel` deadline-alert reuse in `write_dag` * `SerializedDagModel.get_dag_dependencies` * `DagRun.dag_versions` (was sorted by row id) * `DagCode._get_code_from_db` (was ordered by `last_updated`) * `DBDagBag.iter_all_latest_version_dags` (previously had no latest-version filter at all) No schema change and no behaviour change for data where the timestamp order already matched the version order. related: #76041 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) <!-- Generated-by: Qoder following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --> --- ##### Testing * New regression tests for each changed reader, each asserting the version-number result when the timestamp/id order is inverted: * `airflow-core/tests/unit/models/test_dagrun.py::test_dag_run_dag_versions_sorted_by_version_number` * `airflow-core/tests/unit/models/test_dagcode.py::test_latest_code_uses_highest_version_when_last_updated_disagrees` * `airflow-core/tests/unit/models/test_dagbag.py::test_iter_all_latest_version_dags_yields_only_the_latest` * `airflow-core/tests/unit/models/test_serialized_dag.py::test_get_dependencies_uses_latest_version_when_created_at_ordering_disagrees` * `airflow-core/tests/unit/models/test_serialized_dag.py::test_deadline_reuse_uses_latest_version_when_created_at_ordering_disagrees` * `airflow-core/tests/unit/utils/test_db_cleanup.py::test_dag_version_cleanup_keeps_highest_version_number_not_newest_row` * All 10 test IDs pass. Each was confirmed to fail with the source change reverted. * `ruff check` passes on all changed files; `ruff format` applied. * `mypy` reports no errors in the changed files (the only findings are pre-existing environment errors about `fastapi.routing.iter_route_contexts` and a provider stub). * The wider suites around these modules were run locally: all failures there are the same pre-existing `ImportError: cannot import name 'iter_route_contexts' from 'fastapi.routing'` from an outdated fastapi in the local venv, with no assertion failures. CI will cover them properly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
