[
https://issues.apache.org/jira/browse/SPARK-25226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuriy Davygora reopened SPARK-25226:
------------------------------------
> Extend functionality of from_json to support arrays of differently-typed
> elements
> ---------------------------------------------------------------------------------
>
> Key: SPARK-25226
> URL: https://issues.apache.org/jira/browse/SPARK-25226
> Project: Spark
> Issue Type: Improvement
> Components: PySpark, Spark Core
> Affects Versions: 2.3.1
> Reporter: Yuriy Davygora
> Priority: Minor
>
> At the moment, the 'from_json' function only supports a STRUCT or an ARRAY of
> STRUCTS as input. Support for ARRAY of primitives is, apparently, coming with
> Spark 2.4, but it will only support arrays of elements of same data type. It
> will not, for example, support JSON-arrays like
> {noformat}
> ["string_value", 0, true, null]
> {noformat}
> which is JSON-valid with schema
> {noformat}
> {"containsNull":true,"elementType":["string","integer","boolean"],"type":"array"}
> {noformat}
> We would like to kindly ask you to add support for different-typed element
> arrays in the 'from_json' function. This will necessitate extending the
> functionality of ArrayType or maybe adding a new type (refer to
> [[SPARK-25225]])
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
