[ https://issues.apache.org/jira/browse/SPARK-22228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
kant kodali updated SPARK-22228: -------------------------------- Description: {code:java} val inputDS = Seq("""["foo", "bar"]""").toDF {code} {code:java} inputDS.printSchema() root |-- value: string (nullable = true) {code} Input Dataset inputDS {code:java} inputDS.show(false) value ----- ["foo", "bar"] {code} Expected output dataset outputDS {code:java} value ------- "foo" | "bar" | {code} Tried explode function like below but it doesn't quite work {code:java} inputDS.select(explode(from_json(col("value"), ArrayType(StringType)))) {code} and I get the following error {code:java} org.apache.spark.sql.AnalysisException: cannot resolve 'jsontostructs(`value`)' due to data type mismatch: Input schema string must be a struct or an array of structs {code} was: {code:java} val inputDS = Seq("""["foo", "bar"]""").toDF {code} inputDS.printSchema() root |-- value: string (nullable = true) Input Dataset inputDS inputDS.show(false) value ----- ["foo", "bar"] Expected output dataset outputDS value ------- "foo" | "bar" | Tried explode function like below but it doesn't quite work inputDS.select(explode(from_json(col("value"), ArrayType(StringType)))) and I get the following error org.apache.spark.sql.AnalysisException: cannot resolve 'jsontostructs(`value`)' due to data type mismatch: Input schema string must be a struct or an array of structs > Add support for Array<primitive_type> so from_json can parse > ------------------------------------------------------------ > > Key: SPARK-22228 > URL: https://issues.apache.org/jira/browse/SPARK-22228 > Project: Spark > Issue Type: Improvement > Components: Java API > Affects Versions: 2.2.0 > Reporter: kant kodali > > {code:java} > val inputDS = Seq("""["foo", "bar"]""").toDF > {code} > {code:java} > inputDS.printSchema() > root > |-- value: string (nullable = true) > {code} > Input Dataset inputDS > {code:java} > inputDS.show(false) > value > ----- > ["foo", "bar"] > {code} > Expected output dataset outputDS > {code:java} > value > ------- > "foo" | > "bar" | > {code} > Tried explode function like below but it doesn't quite work > {code:java} > inputDS.select(explode(from_json(col("value"), ArrayType(StringType)))) > {code} > and I get the following error > {code:java} > org.apache.spark.sql.AnalysisException: cannot resolve > 'jsontostructs(`value`)' due to data type mismatch: Input schema string must > be a struct or an array of structs > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org