This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 8630652 [SPARK-37575][SQL][FOLLOWUP] Update migration guide for null
values saving in CSV data source
8630652 is described below
commit 8630652f92dc02d5c27f64f81a7803d083817d4e
Author: itholic <[email protected]>
AuthorDate: Wed Dec 15 15:58:24 2021 +0900
[SPARK-37575][SQL][FOLLOWUP] Update migration guide for null values saving
in CSV data source
### What changes were proposed in this pull request?
This is follow-up for https://github.com/apache/spark/pull/34853, to
mention the behavior changes to migration guide, too.
See also https://github.com/apache/spark/pull/34853#issuecomment-994266814
### Why are the changes needed?
We should mention the behavior change to the migration guide, although it's
bug fix.
### Does this PR introduce _any_ user-facing change?
The explanation is added to the migration guide as below:
### How was this patch tested?
Manually built docs
Closes #34905 from itholic/SPARK-37575-followup.
Authored-by: itholic <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
docs/sql-migration-guide.md | 2 ++
1 file changed, 2 insertions(+)
diff --git a/docs/sql-migration-guide.md b/docs/sql-migration-guide.md
index c15f55d..51f3bd3 100644
--- a/docs/sql-migration-guide.md
+++ b/docs/sql-migration-guide.md
@@ -52,6 +52,8 @@ license: |
- Since Spark 3.3, the `strfmt` in `format_string(strfmt, obj, ...)` and
`printf(strfmt, obj, ...)` will no longer support to use "0$" to specify the
first argument, the first argument should always reference by "1$" when use
argument index to indicating the position of the argument in the argument list.
+ - Since Spark 3.3, nulls are written as empty strings in CSV data source by
default. In Spark 3.2 or earlier, nulls were written as empty strings as quoted
empty strings, `""`. To restore the previous behavior, set `nullValue` to `""`.
+
## Upgrading from Spark SQL 3.1 to 3.2
- Since Spark 3.2, ADD FILE/JAR/ARCHIVE commands require each path to be
enclosed by `"` or `'` if the path contains whitespaces.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
