sadpandajoe opened a new pull request, #46079: URL: https://github.com/apache/superset/pull/46079
### SUMMARY `unit-tests (current)` in Python-Unit is sometimes killed at its 45-minute `timeout-minutes` with no test failure, which then fails `unit-tests-required`. Examples: [42271794336](https://github.com/apache/superset/actions/runs/42271794336) on master, and [42323335250 attempt 1](https://github.com/apache/superset/actions/runs/42323335250/attempts/1), which was green on re-run. No specific introducing change. The suite has roughly doubled since it moved to pytest-xdist (about 18.5k → 38k items), and three per-test costs in the test harness grow with it: - Every module-scoped app build runs `SupersetApp.sync_config_to_db`, which re-parses the whole `superset/migrations` alembic `ScriptDirectory`. Caching it takes a build from about 1.3s to 0.5s under coverage, and a run builds about a thousand apps. - Each new fastmcp `Client` lists the tools on its first `call_tool`, and the server's `DereferenceRefsMiddleware` re-inlines the `$ref`s of all 79 tool schemas on every listing. That costs about 0.3s per new client under coverage, across about 2,000 listings per run. - `test_send_rejects_rebound_peer_despite_passed_hostname_check` sleeps through the webhook `backoff` retries (58–113s on CI), because it lacks the `time.sleep` stub its sibling tests use. This change: - `tests/unit_tests/conftest.py`: a session-scoped fixture memoizes `ScriptDirectory.from_config` per configuration. Configs read from an ini file and subclasses fall through to alembic unchanged. - `tests/unit_tests/mcp_service/conftest.py`: dereferenced tool schemas are cached per worker, for the MCP test tree only. An entry is reused only for the same tool object with schemas equal to the ones it was built from, and it is deep-copied on reuse. The patch is installed with `raising=True`, so a fastmcp rename fails loudly. - `webhook_tests.py`: adds the same `time.sleep` stub the sibling tests use. The assertions are unchanged. Test-only: no workflow, timeout, `-n` or product-code change, and no skip/xfail. Coverage is unchanged. Numbers: - Before: on master, the last 25 successful `unit-tests (current)` jobs had a median of 40.6 min and a p90 of 43.4 min. 7 of 406 jobs timed out in the last 30 days. - Local, on the same 2-core/4-thread layout as the runner, with `pytest -n 2 --dist loadfile --cov=superset`: - full `tests/common` + `tests/unit_tests`: 2,014s → 1,375s (−32%), with identical pass/skip/fail counts on all 38,020 items - a 1-in-8 file sample, 3 runs each: median 205s → 162s (−21%) - coverage on that sample is identical: 41.106%, same lines and branches covered - CI after: the job duration from this PR's runs will be added once at least 5 runs have finished. ### TESTING INSTRUCTIONS 1. On this branch, run with `SUPERSET_TESTENV=true SUPERSET_SECRET_KEY=not-a-secret`: ``` time pytest --cov=superset --cov-report= \ tests/unit_tests/mcp_service/dashboard/tool/test_get_dashboard_datasets.py \ tests/unit_tests/reports/notifications/webhook_tests.py ``` Note the pass count and the time. 2. Restore master's versions of the three test files and run the same command again: ``` git checkout master -- tests/unit_tests/conftest.py \ tests/unit_tests/mcp_service/conftest.py \ tests/unit_tests/reports/notifications/webhook_tests.py ``` The pass count is the same, and the run is slower: the webhook test alone takes about 100s. 3. Put the branch versions back with `git checkout HEAD -- <same three files>`. 4. Compare the `unit-tests (current)` job duration on this PR's Python-Unit runs with recent master runs. ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
