dwsmith1983 commented on PR #26228: URL: https://github.com/apache/datafusion/pull/26228#issuecomment-6170149548
That is a fair question, @alamb. Comet is where this came up, and as the PR stands DataFusion itself doesn't rely on the order: HashJoinExec still reports the build side order as not maintained. The order can change on its own today, though. The array map path already returns build rows in input order, but the hash map path returned them last batch first. So the same anti or semi join comes back in a different order depending on how sparse the join keys are. With this change both paths agree. LeftSemi, LeftAnti and LeftMark then emit build rows in input order, so maintains_input_order could report that, and a sort above something like NOT EXISTS ... ORDER BY could be dropped. I can add that with an EXPLAIN test if it seems worthwhile. @andygrove, could you weigh in from the Comet side on whether Comet should depend on this ordering, or whether it's better handled there? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
