jiayuasu commented on PR #1074: URL: https://github.com/apache/sedona-db/pull/1074#issuecomment-5066198110
Correct, I would rather to keep the current behavior as it uses statistics to detect the best side to be the probe side. I think I should correct our discussion today as follows: When there is RS_ZonalStats(vec, ras) operator, assume we have input record batches, each of which is as follows (please adapt this to the DataFusion language). ``` Vec1, Ras1 Vec2, Ras2 Vec3, Ras3 Vec1, Ras2 Vec2, Ras1 ``` If we just perform the operator in the normal order, you will need to either load Ras1 twice or pin Ras1 in memory cache until the end of the query. Instead, we should have a logical / physical optimization rules (or a new algorithm?) to sort the records in the batch by Ras ID. We will have something as follows ``` Vec1, Ras1 Vec2, Ras1 Vec2, Ras2 Vec1, Ras2 Vec3, Ras3 ``` I think this has nothing to do with spatial join. It is a pure zonal stats optimization. I think Kristin might have already done this in his Zonal stats implementation. Please double check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
