mlevkov opened a new issue, #4159: URL: https://github.com/apache/iggy/issues/4159
### Bug description The integration harness launches prebuilt binaries and never builds them, and the documented way to run the tests locally does not build them either. A binary left over from an earlier checkout is used silently, and what an author sees is a large, confusing failure that points at their own changes. `ServerHandle` resolves the server with `Command::cargo_bin(...)` (`core/integration/src/harness/handle/server.rs:936,944`), which only finds a path in the target directory. `ConnectorsRuntimeHandle` and `McpHandle` do the same. Nothing compares the binary against the tree it is being tested with. CI does not have this problem because it builds first. `cargo build --locked --bin iggy-server --bin iggy` appears in `.github/workflows/_test_bdd.yml`, `.github/workflows/coverage-baseline.yml` and several per-language actions. The local instruction in `CLAUDE.md` is `cargo test -p integration -- <path>`, which builds nothing. ### What it looks like I hit this after merging master into a branch. 22 tests across `connectors::random` and `connectors::runtime` failed, including the plain happy-path `random_source_produces_messages`, all with: ``` thread 'main' panicked at core/configs/src/configs_impl/typed_env_provider.rs:328:13: Unknown IGGY_ env var: 'IGGY_PATH'.. Add to IGNORED_ENV_VARS if intentional, or add #[derive(ConfigEnv)] to the config struct. ``` Nothing there is wrong. `IGGY_PATH` is set by the harness itself (`server.rs:952`) and maps to `ServerConfig::path`, which carries `#[derive(ConfigEnv)]`. My `target/debug/iggy-server` was five days old and predated that. `cargo build --bin iggy-server` and all 26 passed. Three things make it expensive rather than merely annoying: - The failure includes the simplest test in the suite, so it reads as "you broke everything" rather than "your binary is old". - The error names a config file and a derive macro, which sends you into the config crate. - CI is green on the same commit, so it looks like a local-only mistake of some other kind. ### Affected area / component Integration tests / test harness. ### Reproduction 1. Build `iggy-server` from some commit. 2. Check out a later commit whose server config has changed. 3. Run `cargo test -p integration -- connectors::random::` without rebuilding. ### Suggested fix Two, and they are independent: 1. **Documentation.** Say in `CLAUDE.md`'s Pitfalls section that the harness launches prebuilt binaries and that CI builds them first, so a local run needs `cargo build --bin iggy-server --bin iggy` after any change that touches a launched binary. That section already collects this class of trap. I will send this one. 2. **A hint in the harness**, if wanted. `TestBinaryError::StartupTimeout` (`server.rs:532`) is the one place that already knows the server did not come up. Including the binary's modification time there turns the message into something an author can act on without guessing. It cannot false-positive, since it only renders on a startup failure. Happy to do this too if a maintainer thinks it is worth the change to shared harness code. Building from the test itself would be the obvious third option and I would advise against it: nextest runs each test in its own process, so concurrent `cargo build` calls would contend on the target lock. ### Contribution - [x] I'm willing to submit a pull request to fix this bug -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
