GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/11464
[SPARK-13543][SQL] Support for specifying compression codec for Parquet/ORC
via option()
## What changes were proposed in this pull request?
This PR adds the support to specify compression codecs for both ORC and
Parquet.
## How was this patch tested?
unittests within IDE and code style tests with `dev/run_tests`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HyukjinKwon/spark SPARK-13543
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11464.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11464
----
commit 9a43e721b0dd57c3b2919aac6d6eead78c38f769
Author: hyukjinkwon <[email protected]>
Date: 2016-03-02T00:10:51Z
Add the support to specify compression codec for Parquet and Orc
commit 9ce0ab707a9e54769b2d55feacd366893a0daf6e
Author: hyukjinkwon <[email protected]>
Date: 2016-03-02T05:26:58Z
Add some tests for testing compressions
commit 59a77d4eac7bd91fbe997d476808a94df66c2ec3
Author: hyukjinkwon <[email protected]>
Date: 2016-03-02T05:27:26Z
Merge upstream
commit 04e4a5167c49b6359f0cfac29d7f2f0c84bd497d
Author: hyukjinkwon <[email protected]>
Date: 2016-03-02T07:27:43Z
Add tests and some comments
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
