I have the following Dataframe in Spark 2.3, coming from a JSON file: root |-- ext_attr: array (nullable = true) | |-- element: struct (containsNull = true) | | |-- key: string (nullable = true) | | |-- value: string (nullable = true)
I would need to convert it to below DataFrame: root |-- ext_attr_map: map (nullable = false) | |-- key: string | |-- value: string I know that in Spark 2.4 I'd use map_from_arrays() SQL function. How can I achieve this in Spark 2.3 ? Please provide a UDF or SQL code. Thanks, Sebastian