kelvinjian-db commented on PR #44716:
URL: https://github.com/apache/spark/pull/44716#issuecomment-1896398723
@beliefer if we have a plan like
```
Filter [isnotnull(sequence(col1, col2, 1)]
+- Join …
+- LocalRelation [col0, col1, col2]
+- …
```
it is possible to have a scenario where:
- if we push down the filter through the join, and then there is a row where
`col1 > col2` in the `LocalRelation`, causing an exception to be thrown
- but if we didn't push down the filter, the join would have filtered out
all the rows where `col1 > col2`, so the exception would not have been thrown
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
