Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21070
> sql("select * from parquetTable where value = '0'")
> sql("select * from parquetTable where value = 0")
Semantically, they are different. The results should be also different. The
plan shows whether these predicates are pushed down or not.
Without the upgrade to 1.10.0, we can't get the benefit from the predicate
pushdown between the above two queries. I also double checked it in my local
environment. Now, it sounds like it is good reason to upgrade the version,
since string predicates are pretty common.
@rdblue @maropu Thank you for your great efforts and patience! This upgrade
is definitely good to have. We should try to make it in the next release.
@maropu Could you also submit a separate PR for your micro benchmark?
Thanks!
cc @cloud-fan @hvanhovell @mswit-databricks Do you have any comment about
the implementation of this PR?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
