ColtenOuO opened a new pull request, #75835: URL: https://github.com/apache/airflow/pull/75835
DB tests run each core test type serially in its own container, with up to 4 types in parallel per job. The heaviest type therefore sets the duration of the job: `Core` (24–29 min), `API` (12–24 min) and `CLI` (10–19 min) kept running alone long after the other types of their slice had finished, leaving most parallel slots idle. This PR: - Carves the slowest folders out of their parent type into companion types that are always selected together with the parent: `PublicAPI` (out of `API`), `CLITask` (out of `CLI`), `Models` and `ModelsTI` (out of `Core`). Selective checks add a companion whenever its parent is selected, so the same tests run as before. - Keeps the `--ignore` args when Non-DB tests combine several test types into one pytest call, so a companion type's tests do not also run in its parent's slice. - Counts only parent types in the CI-minutes estimate (and its calibration script), so the auto-approval estimate does not jump and runs before and after the split stay comparable. - Includes the new types in the default `--parallel-test-types` of `breeze testing core-tests`. - Fixes `test_executor_loader.py` tests that broke when run after CLI tests that reload `executor_loader`: they patched the `ExecutorLoader` class bound at import time instead of the one in the module at run time. Non-DB runs now put `CLI` and `Core` in the same pytest process, which exposed it. #### Measurements Fork CI (GitHub-hosted runners), full test run, two runs of the baseline (A) and two of this change (B), started together on the same `main` commit. Slowest core job per backend, average of two runs: | Backend | A | B | Change | Test step only | |---|---|---|---|---| | MySQL | 31.1 min | 25.1 min | −19% | 25.0 → 17.3 min (−31%) | | Postgres | 31.0 min | 24.0 min | −23% | 25.7 → 16.4 min (−36%) | | Sqlite | 26.6 min | 18.1 min | −32% | 23.6 → 12.7 min (−46%) | | LowestDeps | 27.9 min | 22.4 min | −20% | 24.1 → 17.5 min (−27%) | --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
