coleheflin opened a new pull request, #71637:
URL: https://github.com/apache/airflow/pull/71637
## What
- Fixes 3 `tests_common` files (`test_utils/logs.py`, `api_fastapi.py`,
`providers.py`) that imported `airflow_shared.secrets_masker`/`module_loading`
— a private, never-published distribution only resolvable in a full local
workspace checkout — instead of the vendored `airflow._shared.*` copy
airflow-core bakes into its own installed package (same pattern already used
elsewhere, e.g. this file's own `pytest_plugin.py`).
- Adds a `skip_unless_all_providers_installed` marker (plus a couple of
targeted `pytest.importorskip`s) to 11 tests in
`test_providers_manager.py`/`test_configuration.py` that assumed the full ~100+
provider set is always installed.
- Links the existing TODO in `airflow-core/pyproject.toml` (about eventually
dropping its own hardcoded provider dev-dependencies) to a new tracking issue,
#71641, since it had none.
## Why
addresses (partially): #60770
`uv sync --project airflow-core` (a scoped, core-only sync) succeeds, but
the test suite didn't run clean: collection crashed with `ModuleNotFoundError:
No module named 'airflow_shared'` (it only worked before as a side effect of an
unrelated full workspace sync), and 11 more tests failed on full-provider-set
assumptions. Fixing both gets `pytest airflow-core/tests/unit -m "not db_test"`
passing cleanly in that scoped environment.
(An earlier version fixed the import crash by adding those private packages
as explicit `devel-common` dependencies. That broke the `Compat
3.1.8/3.2.2/3.3.1` CI jobs instead — they deliberately strip local editable
packages before installing a released `apache-airflow` wheel, so the new
dependency reintroduced the same error there. Using the vendored
`airflow._shared.*` path avoids the dependency entirely and survives that
environment too.)
**This only partially addresses #60770.** Its title ("move remaining
provider tests from airflow-core") implies a bigger migration: `airflow-core`'s
own dev group still hardcodes 6 providers (`amazon`, `celery`,
`cncf-kubernetes`, `fab`, `git`, `ftp`) so tests depending on them still pass —
nothing was moved or removed here. Untangling which tests genuinely need which
of those 6, and moving vs. guarding each, is a much larger multi-PR effort, so
it's tracked separately in #71641 rather than folded into this PR. Leaving
#60770 open for a maintainer to decide whether this is enough to close it.
## Testing performed
- Reproduced both failure modes via `uv sync --project airflow-core` +
`pytest`; after the fix, collection is clean (0 errors) and the non-db unit
suite passes fully (0 failed, was 11 failed) — with no new dependency added.
- Restored a full `uv sync` and reran the same files to confirm all tests
still pass for real (marker correctly no-ops when all providers are present).
- Reproduced the CI regression locally (scoped `shared/observability` venv,
and a compat-style env with airflow-core but no standalone shared packages) to
verify the intermediate approach failed and the final one passes.
- Confirmed on CI: all 5 `Compat` jobs pass (including the two that broke on
the intermediate approach), `Shared observability tests` passes, no failures
anywhere.
- Rebased on latest `main`; `prek run --from-ref upstream/main --stage
pre-commit` (ruff, mypy) passes.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Sonnet 5)
Generated-by: Claude Code (Sonnet 5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
