This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.1 by this push: new c23698f [SPARK-37079][PYTHON][SQL] Fix DataFrameWriterV2.partitionedBy to send the arguments to JVM properly c23698f is described below commit c23698ff693ae56dae0064af0940e995b407e9d6 Author: Takuya UESHIN <ues...@databricks.com> AuthorDate: Thu Oct 21 10:25:42 2021 +0900 [SPARK-37079][PYTHON][SQL] Fix DataFrameWriterV2.partitionedBy to send the arguments to JVM properly ### What changes were proposed in this pull request? Fix `DataFrameWriterV2.partitionedBy` to send the arguments to JVM properly. ### Why are the changes needed? In PySpark, `DataFrameWriterV2.partitionedBy` doesn't send the arguments to JVM properly. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually checked whether the arguments are sent to JVM or not. Closes #34347 from ueshin/issues/SPARK-37079/partitionBy. Authored-by: Takuya UESHIN <ues...@databricks.com> Signed-off-by: Hyukjin Kwon <gurwls...@apache.org> (cherry picked from commit 33deeb35f1c994328b577970d4577e6d9288bfc2) Signed-off-by: Hyukjin Kwon <gurwls...@apache.org> --- python/pyspark/sql/readwriter.py | 1 + 1 file changed, 1 insertion(+) diff --git a/python/pyspark/sql/readwriter.py b/python/pyspark/sql/readwriter.py index dd04e0b..9529cf0 100644 --- a/python/pyspark/sql/readwriter.py +++ b/python/pyspark/sql/readwriter.py @@ -1522,6 +1522,7 @@ class DataFrameWriterV2(object): """ col = _to_java_column(col) cols = _to_seq(self._spark._sc, [_to_java_column(c) for c in cols]) + self._jwriter.partitionedBy(col, cols) return self @since(3.1) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org