Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22938#discussion_r235583559
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala
---
@@ -1892,7 +1898,7 @@ class JsonSuite extends QueryTest with
SharedSQLContext with TestJsonData {
.text(path)
val jsonDF = spark.read.option("multiLine", true).option("mode",
"PERMISSIVE").json(path)
- assert(jsonDF.count() === corruptRecordCount)
+ assert(jsonDF.count() === corruptRecordCount + 1) // null row for
empty file
--- End diff --
If that's true, we should not do this. Empty files can be generated in many
cases for now and the behaviour is not currently well defined. If we rely on
this behaviour, it will cause some weird behaviours or bugs hard to fix.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
