Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/22237#discussion_r223849217
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala
---
@@ -554,18 +554,30 @@ case class JsonToStructs(
@transient
lazy val converter = nullableSchema match {
case _: StructType =>
- (rows: Seq[InternalRow]) => if (rows.length == 1) rows.head else null
+ (rows: Iterator[InternalRow]) => if (rows.hasNext) rows.next() else
null
--- End diff --
Actually I am wrong we return empty iterator and as the consequence of that
null in the case if there is no input tokens, there
https://github.com/apache/spark/blob/a8a1ac01c4732f8a738b973c8486514cd88bf99b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala#L398
We can throw `BadRecordException` instead of Nil but this will change
behavior of JSON/CSV datasources.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
