jiayuasu opened a new pull request, #1153: URL: https://github.com/apache/sedona-db/pull/1153
## What changes were proposed in this PR? Applying a geography scalar function (`ST_Area`, `ST_Distance`, `ST_Intersection`, ...) to the output of a spatial join failed with `SedonaCScalarKernelImpl::execute failed: Array input must not be empty`. Two fixes, following the two directions suggested in the issue: 1. **Stop the spatial join from emitting empty batches.** `produce_last_result_batch` unconditionally returned the assembled "unmatched tail rows" batch, which is always zero-row for join types that don't produce unmatched probe rows (e.g. inner joins). So every probe batch whose trailing rows had no match pushed one empty batch downstream. It now suppresses empty batches (`build_joined_batch` is still called for its visited-bitmap bookkeeping). This also helps consumers that reject empty batches, such as lonboard. 2. **Handle zero-length array inputs in `ImportedScalarKernel`.** Imported C scalar kernels (sedona-s2geography) reject zero-length arrays because they broadcast scalar arguments using modulo indexing. The FFI wrapper now short-circuits `num_rows == 0` and returns an empty array of the return type without crossing the C boundary. Empty batches are legal in DataFusion streams, so scalar UDFs must tolerate them regardless of which operator produced them. Note the bug was broader than reported in the issue: joins *with* matches also failed whenever any probe batch had unmatched trailing rows (which is nearly always the case for selective joins), so this broke most geography spatial joins with scalar functions applied on top. ## How was this patch tested? - New unit test `empty_array_input` in `c/sedona-extension/src/scalar_kernel.rs`: an imported kernel that rejects empty input (mimicking the sedona-s2geography kernels) invoked with a zero-length array now returns a zero-length array. Fails without fix 2. - New integration test `test_no_empty_batches_in_join_output` in `rust/sedona-spatial-join`: executes `SpatialJoinExec` partitions directly for a join with matches and a join with zero matches, asserting no emitted batch is empty. Fails without fix 1. - Verified the reproduction from #1084 end to end with a `--features s2geography` build: the selective geography self-join now returns `0, NaN` (matching the empty equi-join behavior), and `ST_Area`/`ST_Intersection`/`ST_Distance` work over join output with and without matches. - Full `sedona-extension` and `sedona-spatial-join` test suites pass. ## Did this PR include necessary documentation updates? - No, this PR does not affect documentation. Fixes #1084. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
