shehabgamin commented on PR #27226: URL: https://github.com/apache/datafusion/pull/27226#issuecomment-6721214982
> I got a question: this change produces a HJ with unbounded input, this seems like a invalid node, as HJ requires to buffer all build-side data before probing, so it can't make progress for unbounded input. > > If symmetric HJ can't handle null-aware semantics, should it return an error instead? @2010YOUY01 Correct, an error is the goal here. A null-aware `NOT IN` can't stream over an unbounded subquery, since a single NULL anywhere changes the answer for every outer row. Before this PR, `JoinSelection` turns it into a `SymmetricHashJoinExec`, which drops the null-aware flag and returns wrong rows. With this change it stays a `HashJoinExec`, and the planner's final sanity check rejects it. So the query now fails at planning instead of returning wrong results. I didn't return an explicit error from `JoinSelection` because the sanity check is already where DataFusion decides whether a plan can run. It already rejects a null-aware join with just one unbounded side this way, so now both cases fail with the same error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
