jiayuasu commented on issue #3153: URL: https://github.com/apache/sedona/issues/3153#issuecomment-5163321436
@paleolimbot @james-willis Thanks. I agree that silently interpreting missing CRS metadata can produce incorrect results, and I agree that type-level CRS tracking is worth exploring. My concern is specifically about enforcing a pairwise one-sided-CRS error when `RS_Intersects`, `RS_Contains`, or `RS_Within` is used as an optimized spatial join condition. SedonaDB and Sedona Spark both allow CRS information to exist at the item level. A raster or geometry column may therefore contain values with different CRSs, including a mixture of values with and without CRS metadata. For scalar predicate evaluation, the proposed rule is straightforward: > Throw an error when exactly one operand in the evaluated pair has CRS metadata. For an optimized spatial join, however, the engine normally uses operand envelopes to eliminate most row pairs before evaluating the exact predicate at the physical planning phase (Cost Base Optimization). If two operands do not have independently identifiable coordinate systems, their envelopes cannot be safely compared before checking their CRS compatibility. To guarantee the pairwise error before CRS-dependent spatial pruning, the engine must first generate the row pairs to which the predicate applies and validate their CRS metadata. With no selective non-spatial join condition, this can degenerate to the `N*M` Cartesian product that the spatial index is intended to avoid. Checking compatibility only after spatial pruning does not enforce the rule consistently: 1. An incompatible pair may have numerically disjoint envelopes. 2. The spatial index removes the pair. 3. The exact predicate is never evaluated. 4. The expected CRS error is never raised. The result is that error visibility can depend on whether the optimizer selects a broadcast-index join, distributed spatial join, or nested-loop execution. Here is my new proposal. The key is to not perform pairwise CRS compatibility validation for optimized raster–vector joins. Instead, each join input can be inspected independently during its normal scan at the physical planning phase (CBO phase): - Record whether any raster value has no CRS object. - Record whether any geometry value has no CRS metadata, such as SRID 0 in Sedona Spark. - If either condition is encountered, emit one warning for the query or spatial operator. - Preserve the existing behavior of interpreting the missing CRS as WGS84. This requires no pairwise comparison and can be incorporated into the normal scan used to build or probe the join. I do not recommend turning this input-level check into an error. That would be substantially stricter than the proposed pairwise rule. A warning is more appropriate (but I am open to it if you guys want to make this an error): > Raster or geometry input is missing CRS metadata. For compatibility, Sedona will interpret the missing CRS as WGS84. Set the raster CRS or geometry SRID explicitly if this is not intended. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
