Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19492#discussion_r144783193
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala
---
@@ -343,6 +367,25 @@ class JacksonParser(
record: T,
createParser: (JsonFactory, T) => JsonParser,
recordLiteral: T => UTF8String): Seq[InternalRow] = {
+ parseWithArrayOfPrimitiveSupport(record, createParser, recordLiteral)
match {
+ case rows: Seq[InternalRow] => rows
+ case _: Seq[_] => throw BadRecordException(() =>
recordLiteral(record), () => None,
+ new RuntimeException("Conversion of array of primitive data is not
yet supported here."))
--- End diff --
What about a user seeing this exception? With the current description
(which I am very open to improve), he/she is aware that he/she is trying to do
something which is not allowed (at least at the moment), ie. we might hit this
exception when using `sqlContext.read.json(...)` on arrays of primitives. Your
suggested description would be a bit weird to a user: he/she might feel he/she
is doing something wrong to achieve something which can be done, but of course
he/she knows nothing about these functions so he/she would be lost IMHO.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
