GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/11593
[SPARK-13728][SQL] Fix ORC PPD test so that pushed filters can be checked.
## What changes were proposed in this pull request?
https://github.com/apache/spark/pull/11509 makes the output only single ORC
file.
It was 10 files but this PR writes only single file. So, this could not
skip stripes in ORC by the pushed down filters.
So, this PR simply repartitions data into 10 so that the test could pass.
## How was this patch tested?
unittest and `./dev/run_tests` for code style test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HyukjinKwon/spark SPARK-13728
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11593.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11593
----
commit 12f8e557caf12b8e9d8dc8b898affbee45ea1b76
Author: hyukjinkwon <[email protected]>
Date: 2016-03-08T10:31:45Z
Fix ORC PPD
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
