HyukjinKwon opened a new pull request, #73:
URL: https://github.com/apache/spark-connect-rust/pull/73
### What changes were proposed in this pull request?
The Coverage workflow (`.github/workflows/coverage.yml`) ran `cargo llvm-cov
-p apache-spark-connect` with **no Spark Connect server** and no `SPARK_REMOTE`
set. The behavioral `e2e_*.rs` tests self-gate on `SPARK_REMOTE`:
```rust
fn should_run() -> bool { std::env::var("SPARK_REMOTE").is_ok() }
#[test] fn dataframe_transforms_and_setops() { if !should_run() { return; }
... }
```
So in the coverage job those tests **no-op**, and the runtime execution
paths (collect/count/RPC-backed DataFrame, functions, session, catalog, ...)
are never exercised. Only unit + golden (plan-serialization) tests run, so the
Rust coverage badge reported **~73.3%**, well below the real coverage. Those
same e2e tests already run in `build_python_connect.yml`, which downloads Spark
4.2.0, starts a Connect server, and sets `SPARK_REMOTE`.
This PR makes the coverage job start a Spark 4.2.0 Connect server (mirroring
`build_python_connect.yml`) and measure coverage in two accumulated
`cargo-llvm-cov` passes that are then combined:
1. the default unit + golden tests, run in parallel;
2. the behavioral e2e suite against the live server (`SPARK_REMOTE` set),
run serially since the tests share one server session.
`--no-report` accumulates raw coverage from each pass; `cargo llvm-cov
report` merges both into one JSON summary consumed by the existing badge parser.
Supporting changes: JDK 17 + the Python deps the server's workers need, a
shared Spark-dist cache key (reused from the parity job), a 45-minute job
timeout, and a server-log artifact for debugging. The two duplicate Python
setup steps are consolidated into one.
### Why are the changes needed?
So the Rust coverage badge reflects the coverage actually achieved when the
runtime paths are exercised, instead of undercounting because the e2e tests
silently no-op without a server.
### Does this PR introduce _any_ user-facing change?
No. CI-only change.
### How was this patch tested?
CI (the coverage workflow itself).
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
