james-willis opened a new pull request, #1145: URL: https://github.com/apache/sedona-db/pull/1145
## What Adds `chunk_index: List<Int64> NULLABLE` to the Band schema: one entry per `dim_names` entry, the zero-based block coordinate this band occupies along that axis within a larger logical array it's one chunk of — a chunk *number*, not a pixel offset, so it stays meaningful across uneven/remainder chunk sizes. Nullable at the schema level, same as `crs`/`transform` on `Raster` today: unset for a band's ordinary role inside `Raster.bands` (addressed by index/name, not block coordinate), required and validated by whichever accessor or function actually depends on it for a standalone chunked-table (DataArray-style) use case — a chunked table with one row per block, where each chunk currently has nowhere to carry its own position except external sibling columns the caller has to build and maintain by hand. ## Why A chunk keeps its own block coordinate wherever it goes — through a `SELECT`, out of a function's return value, across a join, eventually through `UNNEST` — instead of losing it the moment it's separated from the specific row it started in. Carrying this on the value itself, rather than requiring a consumer to invent and maintain separate `dim_0_idx`/`dim_1_idx`-style columns alongside it, means the block-coordinate alignment a chunked-array join needs can match directly on `chunk_index` with no dependency on external bookkeeping. ## What's in it - `RasterSchema`: new `chunk_index_type()`, the field appended to `band_type()`, `column::CHUNK_INDEX`/`band_indices::CHUNK_INDEX`, `band_indices::FIELD_COUNT` bumped 9→10. - `RasterBuilder`: `StartBandArgs.chunk_index` (validated against `dim_names.len()`), threaded through `start_band`/`finish` following the same values+offsets+validity pattern already used for `source_shape`/`view`. - `BandRef`: new `chunk_index()` accessor (default `None`), a real implementation in the Arrow-backed `BandRefImpl`. - `BandRef::copy_into`: `BandOverrides.chunk_index`, inherited by default (`overrides.chunk_index.or_else(|| self.chunk_index())`) — same pattern as `nodata`/`outdb_uri`/`outdb_format` — so a chunk's position survives a derive operation, not just a raw copy. All ~30 existing `StartBandArgs` construction sites are unaffected: every one already uses `..StartBandArgs::new(...)` struct-update syntax, so the new field's `None` default requires no changes there — verified directly by compiling the whole workspace, not assumed. ## Verification - New tests: round-trip (present and absent), a mixed-validity case across multiple bands in one array (the case most likely to catch an offset/validity-bitmap bug), length-mismatch validation, an Arrow IPC serialize/deserialize round-trip, and `copy_into`'s inherit-by-default + explicit-override paths. - `cargo test` for `sedona-schema`, `sedona-raster`, `sedona-raster-functions`, `sedona-raster-zarr`, `sedona-spatial-join-raster`, `sedona-testing` — all pass, no regressions. - `cargo clippy --all-targets -- -D warnings` and `cargo fmt --all -- --check` — clean. - Full workspace `cargo test` (excluding the environment-only `gdal-sys`/`sedona-gdal` bindgen issue on this machine, unrelated to this change) — clean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
