paleolimbot opened a new issue, #605: URL: https://github.com/apache/sedona-db/issues/605
In #72 we introduced a barrier function as a workaround to ensure there was a clear path to unambiguous execution of a knn join + a filter. This is needed because a filter followed by a KNN join is not the same as a KNN join followed by a filter in the same way as a regular join, and DataFusion has several optimizer rules that attempt to optmize filters in a way that is not strictly correct for a KNN join. After #562 we now plan spatial joins using a logical optimizer rule! This means that we can opt out of join optimizer rules as long as the LogicalPlan that is seen by DataFusion's optimizer rules contains the Spatial Join logical operator instead of a regular join with an `ST_KNN()` condition. We could do this either by pre-processing a LogicalPlan before it is passed to DataFusion's optimizer, by allowing KNN joins to be created directly by the R/Python DataFrame API, or by inserting an optimizer rule at the beginning of the rule list instead of at the end. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
