james-willis opened a new pull request, #1233:
URL: https://github.com/apache/sedona-db/pull/1233
Parity coverage for `RS_ZonalStats` and `RS_ZonalStatsAll`, probe-first, one
module per function. Branches off `main`, independent of the other open parity
PRs.
### What agrees (32 passing tests, every anchor exact)
The headline probe finding: **every statistic matches numpy's exact double
bit-for-bit on both engines** — including mean, variance, and stddev — so there
is no tolerance anywhere in the module. The shared conventions the anchors pin:
- sample (ddof=1) variance and stddev;
- even-count median averages the two middle values; odd-count is the exact
middle element;
- mode ties break toward the **higher** value (probed with planted 2-vs-2
ties);
- `avg`/`average`/`sd` aliases accepted by both;
- `exclude_no_data` defaults true (a planted in-roi nodata pixel drops from
the count) and can be disabled;
- `all_touched` selects a centre-free sliver that the default centre-in rule
ignores;
- a disjoint roi yields NULL under the default lenient behavior;
- both engines refuse strict-disjoint (`lenient=false`), unknown statistics,
and out-of-range bands (0 and beyond) — contrast the `RS_BandNoDataValue`
getter, where SedonaDB returns NULL for out-of-range bands.
`RS_ZonalStatsAll` is compared field by field — `s['field']` parses in both
dialects — because the whole-struct schemas differ (SedonaDB types `count` as
nullable Int64; Sedona Spark returns non-nullable Doubles throughout) and the
harness stringifies scalar columns, not structs.
### What diverges (3 xfails)
- **Band-less forms on a multi-band raster** (both functions): SedonaDB
requires the band argument (documented as deliberate in the kernel); Sedona
Spark defaults to band 1.
- **CRS-carrying roi on a CRS-less raster**: SedonaDB refuses ("geometry has
a CRS but the raster does not"); Sedona Spark computes as if they matched.
### Verification
`integration/spark-parity` locally: the two modules run `32 passed, 3
xfailed` (pyspark 4.0.4, Sedona 1.9.1); CI runs the parity lane on this branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
