GitHub user maropu opened a pull request:
https://github.com/apache/spark/pull/11324
[SPARK-11691][SQL] Support setting hadoop compression codecs in
DataFrameWriter#option
## What changes were proposed in this pull request?
This pr is to support hadoop compression codecs when saving DataFrame to
disk.
This is rework from #9657 because it gets stale.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/maropu/spark pr9657
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11324.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11324
----
commit b1e94132b75dc5d13d55c82ea9ca7422da45a04f
Author: Jeff Zhang <[email protected]>
Date: 2015-11-12T08:18:46Z
[SPARK-11691][SQL] Allow to specify compression codec in HadoopFsRelation
when saving
commit bf14b49dad375c361b8aaab87315bf3d2c108032
Author: Jeff Zhang <[email protected]>
Date: 2015-11-12T09:50:28Z
fix compilation issue
commit f014ab253e1f1ce277b5a910ff19f6a155357fbd
Author: Jeff Zhang <[email protected]>
Date: 2015-12-02T03:24:32Z
address comments
commit 58f26bab59b203435b5654b286ab90efe8964413
Author: Jeff Zhang <[email protected]>
Date: 2015-12-02T05:03:08Z
minor change
commit 4e453884c75ef15c2281bae0dad0bb87e0272504
Author: Jeff Zhang <[email protected]>
Date: 2015-12-03T05:20:07Z
change back to 1.7.0
commit 7548032cf754d6f847ac493a307a5e838cd9cac7
Author: Jeff Zhang <[email protected]>
Date: 2016-01-29T04:57:24Z
fix review comments
commit f6723322605bbd62ae55713e46303daa1cfc885b
Author: Jeff Zhang <[email protected]>
Date: 2016-01-29T05:36:14Z
fix code style
commit f290a133040561f7efa8f57e063f765d34c0f1fa
Author: Jeff Zhang <[email protected]>
Date: 2016-01-29T08:35:01Z
minor fix
commit b9efe53fab30290f5b82a416403e2c8c8e13cc68
Author: Takeshi YAMAMURO <[email protected]>
Date: 2016-02-23T16:19:54Z
Apply comments
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
