Dandandan opened a new pull request #9937: URL: https://github.com/apache/arrow/pull/9937
This PR adds a (ignored) test for https://issues.apache.org/jira/browse/ARROW-12266 ``` SELECT id1, id2 FROM (SELECT null AS id1) t1 LEFT JOIN (SELECT 0 AS id2) t2 ON id1 = id2 ``` current result: ```NULL, NULL``` (should be empty result set) We should filter on nulls beforehand to make this result correct. Probably the best way to go here I think is to add a filter in the logical plan on non-null for inner / left and right joins. This can make things more efficient as the non-null filter can be pushed down which can lead to efficiency gains (making data-set smaller, not having to deal with nullable data in batches, or even entire files could be skipped when they only contain nulls). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
