xudong963 commented on issue #1321: URL: https://github.com/apache/arrow-datafusion/issues/1321#issuecomment-971789330
> Looking at the logical plan I believe this is due to FilterPushDown (though I'm not familiar with the optimizer yet). > I think we should make an exception for `IS NULL` and `IS NOT NULL` filter because `LEFT JOIN` also results in `NULL` for every missing value on the join column of the right table, which should be included in the end result. Yes, you are right. We need a special handle `NULL` in `FilterPushDown`, more specific in https://github.com/apache/arrow-datafusion/blob/bcd12589ee8f428cd860e0eefd3f225ed6293817/datafusion/src/optimizer/filter_push_down.rs#L250. (LOL, I have to say, null is a real pain in the neck in DBMS) BTW, `RIGHT JOIN` also has the bug. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
