GitHub user kevinyu98 opened a pull request:
https://github.com/apache/spark/pull/10299
[SPARK-12231][SQL]create a combineFilters' projection when we call
buildPartitionedTableScan
Hello Michael & All: Here I am submitting another approach to solve this
problem. Can you verify ?
I think the problem is related to change from spark-10829,
before that PR change, the projects and filters are done inside
buildPartitionedTableScan.
With that PR change, the filter expression divide to 3 parts, the filter
left outside of the scan (combineFilters ) needs a different projection.
So the fix is to create a combine projection for the outside filter and
beyond.
Thanks for your comments.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kevinyu98/spark working_on_spark-12231
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10299.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10299
----
commit 2333c6d3dffd580529705e33f5ccdc8871670c0f
Author: Kevin Yu <[email protected]>
Date: 2015-12-14T19:51:35Z
another approach to fix this problem
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
