Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22787#discussion_r226884659
--- Diff: docs/sql-migration-guide-upgrade.md ---
@@ -11,6 +11,8 @@ displayTitle: Spark SQL Upgrading Guide
- In PySpark, when creating a `SparkSession` with
`SparkSession.builder.getOrCreate()`, if there is an existing `SparkContext`,
the builder was trying to update the `SparkConf` of the existing `SparkContext`
with configurations specified to the builder, but the `SparkContext` is shared
by all `SparkSession`s, so we should not update them. Since 3.0, the builder
comes to not update the configurations. This is the same behavior as Java/Scala
API in 2.3 and above. If you want to update them, you need to update them prior
to creating a `SparkSession`.
+ - In Spark version 2.4 and earlier, the parser of JSON data source
treats empty strings as null for some data types like `IntegerType`. For
`FloatType` and `DoubleType`, it fails on empty strings and throws exceptions.
Since Spark 3.0, we disallow empty strings and will throw exceptions for data
types except for `StringType` and `BinaryType`.
--- End diff --
Thanks. I've updated it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
