Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/17294
@hvanhovell It would not be trivial to backport to 2.1/2.0. The correlated
predicates in the NOT IN subquery have been pulled up in Analyzer phase and
combined with the the columns in the LHS and RHS of NOT IN. We will need a
different technique from this fix such as marking the correlated predicates
differently from the columns in NOT IN.
NOT IN is not commonly used with null values. I think the effort may not be
worth it.
Thank you for your quick turnaround time in the review and merging the PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
