kant kodali created SPARK-22228:
-----------------------------------
Summary: Add support for Array<primitive_type> so from_json can
parse
Key: SPARK-22228
URL: https://issues.apache.org/jira/browse/SPARK-22228
Project: Spark
Issue Type: Improvement
Components: Java API
Affects Versions: 2.2.0
Reporter: kant kodali
[val inputDS = Seq("""["foo", "bar"]""").toDF]
inputDS.printSchema()
root
|-- value: string (nullable = true)
Input Dataset inputDS
inputDS.show(false)
value
-----
["foo", "bar"]
Expected output dataset outputDS
value
-------
"foo" |
"bar" |
Tried explode function like below but it doesn't quite work
inputDS.select(explode(from_json(col("value"), ArrayType(StringType))))
and I get the following error
org.apache.spark.sql.AnalysisException: cannot resolve 'jsontostructs(`value`)'
due to data type mismatch: Input schema string must be a struct or an array of
structs
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
