cmcfarlen commented on PR #13545:
URL: https://github.com/apache/trafficserver/pull/13545#issuecomment-5482014182
I got the Uranium suite running natively on macOS (no container) and wired
it into neovim's `neotest-python`, and hit a handful of small friction points
along the way. All of these are optional polish rather than blockers — the
suite itself worked well: **69 passed / 2 failed / 4 skipped** across
`uranium_tests/cache` and `uranium_tests/basic` on a native darwin-arm64 build.
## The issue I'd most like to see fixed: unresolved imports in editors
Opening any `test_*.py` under `tests/uranium_tests/` in an LSP-backed editor
reports `Import "pytest" could not be resolved`, because pyright has no idea
where the environment lives. Anyone opening a Uranium test in VS Code, neovim,
or PyCharm gets import noise until they work this out for themselves.
The support-package paths are repo-invariant, so most of it can just be
checked in. Adding this to `tests/pyproject.toml` fixes the `extraPaths` half
for everyone:
```toml
[tool.pyright]
extraPaths = [".", "uranium_tests/remap", "uranium_tests/remap_yaml",
"uranium_tests/lib"]
```
`tests/` is already pyright's root (nearest ancestor with `pyproject.toml`),
so it lands in the right scope with no per-developer configuration. Only the
venv location stays machine-local, and developers who `uv sync` in-tree
wouldn't need to set even that.
One caveat if you take this: a `pyrightconfig.json` overrides
`[tool.pyright]` entirely, so it's worth mentioning in the docs which one wins.
## Suggestions that would make the pytest setup easier
### 1. Let a bare `pytest` invocation work
This is the biggest ergonomic win. Right now a front-end other than
`urtest.sh` has to reproduce, exactly: `-p tools.uranium.plugin`,
`--import-mode=importlib`, and a four-entry `PYTHONPATH`. Everything else
already has an environment-variable fallback (`ATS_BIN`, `PROXY_VERIFIER_BIN`,
`ATS_BUILD_ROOT`, `ATS_URTEST_SANDBOX` in `plugin.py:get_runtime`) — which is
great, and is what made this tractable — but plugin registration and `sys.path`
don't.
A rootdir `tests/conftest.py` would close the gap:
```python
import sys
from pathlib import Path
_TESTS = Path(__file__).parent
sys.path[:0] = [str(_TESTS), *(str(_TESTS / "uranium_tests" / name) for name
in ("remap", "remap_yaml", "lib"))]
pytest_plugins = ["tools.uranium.plugin"]
```
`tests/` is the rootdir (inifile is `tests/pyproject.toml`), so
`pytest_plugins` is legal there under pytest 8+. After that, any IDE test
runner, `neotest`, or plain `pytest -k foo` collects and runs with only the
four path options — no wrapper-specific knowledge.
### 2. Move `--import-mode=importlib` into the ini
```toml
[tool.pytest.ini_options]
addopts = "--import-mode=importlib"
```
One less flag every front-end has to know about, and it documents the
requirement in the obvious place.
### 3. Don't depend on `uv run` for helper-process resolution
`microdns` and `microserver` are console scripts in the venv, exec'd by bare
name — `replay.py:135`, `service_factory.py:104`, `service_factory.py:176`.
That only works because `uv run` puts `.venv/bin` on `PATH`. Invoke the same
pytest through any other launcher and it fails with:
```
FileNotFoundError: [Errno 2] No such file or directory: 'microdns'
```
which is a confusing failure to land on, since the package is plainly
installed. Resolving them relative to the running interpreter would make the
suite launcher-agnostic:
```python
shutil.which("microdns", path=str(Path(sys.executable).parent)) or "microdns"
```
### 4. `LD_LIBRARY_PATH` is a no-op on macOS
`runner.py:171` sets `LD_LIBRARY_PATH`, which dyld ignores; the darwin
equivalent is `DYLD_LIBRARY_PATH` (or `DYLD_FALLBACK_LIBRARY_PATH`). Since
`find_container_runtime` now prefers Apple container on darwin and
`--no-run-in-container` makes native macOS runs a supported path, this is worth
conditioning on `sys.platform`. It didn't bite me because the install carries
usable rpaths, but it would for anyone whose build doesn't.
### 5. `tests/uv.lock` doesn't reach the build tree
`tests/CMakeLists.txt` does `configure_file(pyproject.toml pyproject.toml
COPYONLY)` but has no rule for `uv.lock`, so `uv sync` in `<build>/tests`
resolves fresh and writes its own lock. That means the committed lockfile never
constrains the environment any run actually uses — including CI's.
It's already drifted. Comparing the committed lock against one generated in
a build tree: same 53 packages, 8 at different versions.
```
charset-normalizer 3.5.0 -> 3.5.1
cryptography 50.0.0 -> 50.0.1
grpcio 1.83.0 -> 1.83.1
grpcio-tools 1.83.0 -> 1.83.1
gunicorn 26.0.0 -> 26.2.0
idna 3.18 -> 3.19
```
`cryptography` and `grpcio` moving underneath the TLS and gRPC tests is
exactly the class of drift a lockfile is meant to prevent. Adding `uv.lock` to
the `configure_file` set (and `--frozen` or `--locked` on the sync) would make
build-tree environments reproducible.
## Minor doc note
`tests/urtest.sh` defaults to container execution, which is the right
default, but the docs could state plainly that `--no-run-in-container` against
a natively configured build works and is a reasonable macOS workflow — a large
majority of tests pass there. The paths a front-end needs are all baked into
the generated `<build>/tests/urtest.sh` as `--key=value` pairs, which turned
out to be a very convenient integration point; worth keeping stable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
