james-willis opened a new pull request, #1217:
URL: https://github.com/apache/sedona-db/pull/1217
## What
Wires the `integration/spark-parity` pytest suite (added in #1159) into the
`python` workflow, answering the reviewer question on #1203: "Are they running
in CI now?" — with this PR, yes.
Changes, all in `.github/workflows/python.yml` plus a one-sentence README
update:
- `integration/spark-parity/**` added to the `pull_request` paths filter.
- New steps appended to the existing `test` job, after the current test
steps (the job already builds and installs sedonadb editable, which the suite
imports):
- `actions/setup-java@v6` (temurin 17) — the JVM Sedona Spark needs.
- `pip install "pyspark==4.1.*" apache-sedona` — the newest Spark minor
Sedona 1.9.1 ships an artifact for (unpinned resolves pyspark 4.2, whose
FunctionRegistry rejects the 4.1 jar).
- An `actions/cache@v6` step for the Ivy jar directory
(`~/.ivy2-spark-parity`).
- `python -m pytest -vv` run from `integration/spark-parity`, with
`SEDONADB_SPARK_IVY_DIR` pointed at the cached directory.
- `integration/spark-parity/README.md`: the "not wired into CI" sentence now
says the suite runs in the python workflow.
## Cache design
The cache key is `spark-parity-jars-${{ runner.os }}-${{
hashFiles('python/sedonadb/python/sedonadb/testing_spark.py') }}`.
`testing_spark.py` is the file that pins `SEDONA_SPARK_VERSION` and
`GEOTOOLS_WRAPPER_VERSION` — the exact Maven coordinates Ivy resolves — so the
cache invalidates precisely when the jar pins change and never goes stale on a
version bump. `SEDONADB_SPARK_IVY_DIR` is set to the same path the cache step
saves, so warm runs skip the Maven download entirely.
## Verification
The workflow triggers on changes to `.github/workflows/python.yml`, so this
PR's own CI run exercises the new lane end-to-end.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
