Repository: spark Updated Branches: refs/heads/master 087879a77 -> 33d43bf1b
[SPARK-22484][DOC] Document PySpark DataFrame csv writer behavior whe⦠## What changes were proposed in this pull request? In PySpark API Document, DataFrame.write.csv() says that setting the quote parameter to an empty string should turn off quoting. Instead, it uses the [null character](https://en.wikipedia.org/wiki/Null_character) as the quote. This PR fixes the doc. ## How was this patch tested? Manual. ``` cd python/docs make html open _build/html/pyspark.sql.html ``` Author: gaborgsomogyi <gabor.g.somo...@gmail.com> Closes #19814 from gaborgsomogyi/SPARK-22484. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/33d43bf1 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/33d43bf1 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/33d43bf1 Branch: refs/heads/master Commit: 33d43bf1b6f55594187066f0e38ba3985fa2542b Parents: 087879a Author: gaborgsomogyi <gabor.g.somo...@gmail.com> Authored: Tue Nov 28 10:14:35 2017 +0900 Committer: hyukjinkwon <gurwls...@gmail.com> Committed: Tue Nov 28 10:14:35 2017 +0900 ---------------------------------------------------------------------- python/pyspark/sql/readwriter.py | 3 +-- .../src/main/scala/org/apache/spark/sql/DataFrameWriter.scala | 3 ++- 2 files changed, 3 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/33d43bf1/python/pyspark/sql/readwriter.py ---------------------------------------------------------------------- diff --git a/python/pyspark/sql/readwriter.py b/python/pyspark/sql/readwriter.py index a75bdf8..1ad974e 100644 --- a/python/pyspark/sql/readwriter.py +++ b/python/pyspark/sql/readwriter.py @@ -828,8 +828,7 @@ class DataFrameWriter(OptionUtils): set, it uses the default value, ``,``. :param quote: sets the single character used for escaping quoted values where the separator can be part of the value. If None is set, it uses the default - value, ``"``. If you would like to turn off quotations, you need to set an - empty string. + value, ``"``. If an empty string is set, it uses ``u0000`` (null character). :param escape: sets the single character used for escaping quotes inside an already quoted value. If None is set, it uses the default value, ``\`` :param escapeQuotes: a flag indicating whether values containing quotes should always http://git-wip-us.apache.org/repos/asf/spark/blob/33d43bf1/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---------------------------------------------------------------------- diff --git a/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala b/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala index e3fa2ce..35abecc 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala @@ -592,7 +592,8 @@ final class DataFrameWriter[T] private[sql](ds: Dataset[T]) { * <li>`sep` (default `,`): sets the single character as a separator for each * field and value.</li> * <li>`quote` (default `"`): sets the single character used for escaping quoted values where - * the separator can be part of the value.</li> + * the separator can be part of the value. If an empty string is set, it uses `u0000` + * (null character).</li> * <li>`escape` (default `\`): sets the single character used for escaping quotes inside * an already quoted value.</li> * <li>`escapeQuotes` (default `true`): a flag indicating whether values containing --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
