GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/21712
[SPARK-22384][SQL][followup] Refine partition pruning when attribute is
wrapped in Cast
## What changes were proposed in this pull request?
As mentioned in https://github.com/apache/spark/pull/21586 ,
`Cast.mayTruncate` is not 100% safe, string to boolean is allowed. Since
changing `Cast.mayTruncate` also changes the behavior of Dataset, here I
propose to add a new `Cast.canSafeCast` for partition pruning.
## How was this patch tested?
new test cases
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark safeCast
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21712.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21712
----
commit 7b316ff38486ad42b830cfed5d28b386485e3e6f
Author: Wenchen Fan <wenchen@...>
Date: 2018-07-04T07:44:48Z
only support cast in partition predicate when it's safe
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
