james-willis opened a new issue, #1078: URL: https://github.com/apache/sedona-db/issues/1078
When planning a raster–vector spatial join, the choice of which operand to index on the build side vs stream on the probe side does not yet use the knowledge that one operand is a raster and the other a vector. The raster operand is typically the smaller input with far fewer vertices (one footprint per raster), so streaming the raster / indexing the vector is often preferable — and it avoids buffering (and, on the build side, gc-compacting/copying) the raster's potentially large in-DB band payloads. But this interacts with input cardinality: when the vector table is the vastly larger input (few raster tiles × billions of geometries), forcing the vector onto the fully-buffered build side can blow up memory, which is exactly what the statistics-based side heuristic is designed to avoid. Options to weigh: - Pin the raster operand to the probe side, but conditionally (on relative input sizes) rather than unconditionally, so a huge vector table is not forced onto the build side. - Make build-side ingest share raster view buffers instead of compacting/copying them, removing the copy cost on whichever side the raster lands and keeping the statistics heuristic intact. - Pair with a loader-level cache so a raster matching many vectors is loaded once regardless of side. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
