Kontinuation opened a new pull request, #521: URL: https://github.com/apache/sedona-db/pull/521
This refactor makes `BuildSideBatchesCollector ` and `SpatialJoinStream` work with streams of `EvaluatedBatch`es instead of directly with streams of `RecordBatch`es. We will have `EvaluateBatch`es read directly by spill readers so adding this layer of abstraction make the main part of spatial join care less about whether the stream is directly from the source of read from spill files. The stream for the build-side automatically compact batches to avoid holding large sparse binary view arrays in memory. The `EvaluateOperandBatchStream` performs the batch compaction automatically before evaluating the operands of spatial predicates. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
