jiayuasu opened a new issue, #134:
URL: https://github.com/apache/sedona-spatialbench/issues/134
Follow-up to #131, which adds the geography query suite as SQL and docs
only. The benchmark harness, the CI matrix and `benchmark/answers/` are
untouched by it, so the geography queries are defined but never timed or
verified.
## Why the two suites need separate answers
`verify_results.py` resolves ground truth per *query*:
`benchmark/answers/sf<n>/q<n>.parquet`. The geography suite reuses the query
IDs `q1..q12` but returns **different values for the same IDs** — a 0.45°
planar radius and a 50 000 m geodesic radius simply select different rows. So
the answers cannot be shared, and a per-suite (or per-engine) answer path is
needed rather than a flat per-query one.
Concretely, at SF1: Q1 returns a different 100 rows, Q9's areas are m²
instead of square degrees, and Q7's `line_distance_m` is a real geodesic length
rather than degrees divided by `0.000009`. Q2 (541) and Q4 (258 zones,
identical counts and order) happen to agree with the planar answers, which
makes them useful cross-checks but does not generalise.
## Work involved
1. **Runner** — teach `benchmark/run_benchmark.py` to select the suite.
Either a new engine key (`sedonadb_geography`, which keeps the existing
`<engine>_<query>_result.csv` dump naming working unchanged) or a `--suite
{geometry,geography}` flag. `get_sql_queries()` currently hardcodes the two
geometry dialect classes.
2. **Answers** — `benchmark/answers/geography/sf{1,10}/q<n>.{csv,parquet}`,
generated the same way as the existing ones, plus a note in
`benchmark/answers/README.md` explaining why the two sets differ.
3. **Verify** — `verify_results.py` needs a per-engine (or per-suite)
answer-directory mapping, and its `LIMIT_QUERIES` boundary-tie set should be
re-derived for the geography suite rather than assumed identical.
4. **CI** — a geography job block in `.github/workflows/benchmark.yml`,
mirroring the existing per-query matrix. Note the workflow already triggers on
`spatialbench-queries/**`, so #131 runs the full geometry matrix without
exercising any of the new SQL.
5. **Summarize** — `summarize_results.py` has a hardcoded engine list and
icon map.
`QUERY_COUNT = 12` is duplicated across `run_benchmark.py`,
`verify_results.py`, `generate_answers.py` and `summarize_results.py`, and the
CI matrix lists `q1..q12` literally per engine; the geography suite carries 11
queries (no `ST_KNN` yet — apache/sedona-db#1086, apache/sedona#3178), so
whichever mechanism is chosen has to tolerate a suite with a gap rather than
assume a contiguous 1..N.
## Blocked on
Two SedonaDB defects would corrupt any answers generated today:
- **apache/sedona-db#1084** — Q9 cannot complete: a geography scalar
function over geography spatial-join output fails with `Array input must not be
empty`.
- **apache/sedona-db#1085** — one zone polygon matches points anywhere on
Earth, inflating Q4, Q6, Q10 and Q11.
See also #133 for committing the notebook's executed outputs, which is
blocked on the same two.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
