Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/22597
In `ParquetFilter`, the way we test if a predicate pushdown works is by
removing that predicate from Spark SQL physical plan, and only relying on the
reader to do the filter. Thus, if there is a bug in pushdown filter in reader,
Spark will get the incorrect result. This can use in test to ensure no
regression later.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
