Github user mswit-databricks commented on a diff in the pull request:
https://github.com/apache/spark/pull/20694#discussion_r172142857
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala
---
@@ -680,4 +681,31 @@ class JsonExpressionsSuite extends SparkFunSuite with
ExpressionEvalHelper {
)
}
}
+
+ test("from_json missing fields") {
+ val conf = SQLConf.get
+ for (forceJsonNullableSchema <- Seq(false, true)) {
+ conf.setConf(SQLConf.FROM_JSON_FORCE_NULLABLE_SCHEMA,
forceJsonNullableSchema)
--- End diff --
I tried moving the test, but it is not as easy as it may seem.
`org.apache.spark.sql.execution.datasources.json.JsonSuite` lacks
`checkEvaluation` that I'm using here. In general,
`org.apache.spark.sql.execution.datasources.json.JsonSuite` seems to be meant
for things such as `spark.read.json()` and not the `from_json()` function, so
the test would be misplaced there. I think that it's better to keep the test in
`JsonExpressionsSuite` and revert the config.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
