Repository: spark Updated Branches: refs/heads/branch-2.0 3cc258efb -> 809af6d9d
[TRIVIAL] [PYSPARK] Clean up orc compression option as well ## What changes were proposed in this pull request? This PR corrects ORC compression option for PySpark as well. I think this was missed mistakenly in https://github.com/apache/spark/pull/13948. ## How was this patch tested? N/A Author: hyukjinkwon <gurwls...@gmail.com> Closes #13963 from HyukjinKwon/minor-orc-compress. (cherry picked from commit d8a87a3ed211dd08f06eeb9560661b8f11ce82fa) Signed-off-by: Davies Liu <davies....@gmail.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/809af6d9 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/809af6d9 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/809af6d9 Branch: refs/heads/branch-2.0 Commit: 809af6d9d7df17f5889ebd8640c189e8d1e143a8 Parents: 3cc258e Author: hyukjinkwon <gurwls...@gmail.com> Authored: Wed Jun 29 13:32:03 2016 -0700 Committer: Davies Liu <davies....@gmail.com> Committed: Wed Jun 29 13:32:35 2016 -0700 ---------------------------------------------------------------------- python/pyspark/sql/readwriter.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/809af6d9/python/pyspark/sql/readwriter.py ---------------------------------------------------------------------- diff --git a/python/pyspark/sql/readwriter.py b/python/pyspark/sql/readwriter.py index 44bf744..78d992e 100644 --- a/python/pyspark/sql/readwriter.py +++ b/python/pyspark/sql/readwriter.py @@ -695,8 +695,7 @@ class DataFrameWriter(OptionUtils): self.mode(mode) if partitionBy is not None: self.partitionBy(partitionBy) - if compression is not None: - self.option("compression", compression) + self._set_opts(compression=compression) self._jwrite.orc(path) @since(1.4) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org