[
https://issues.apache.org/jira/browse/SPARK-51975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-51975:
-----------------------------------
Labels: pull-request-available (was: )
> Support variant_from_arrays and variant_from_entries
> -----------------------------------------------------
>
> Key: SPARK-51975
> URL: https://issues.apache.org/jira/browse/SPARK-51975
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Affects Versions: 4.1.0
> Reporter: Daniël Tom
> Priority: Major
> Labels: pull-request-available
>
>
> Currently, Spark requires a two-step process to create Variant maps from
> arrays:
> # Create MapType using map_from_arrays/map_from_entries
> # Convert to Variant using to_variant_object()
> A direct variant_from_arrays/variant_from_entries implementation reduces
> memory usage, and improves CPU efficiency. Particularly benefit workloads
> with large map structures or high-volume of data.
> {*}Example{*}:
> ||Function||Description||Example||
> |variant_from_entries|Transforms an array of key-value pair entries (structs
> with two fields) into a variant object|{code:java}
> SELECT variant_from_entries(array(struct('1', 'a'), struct('2',
> 'b')));{code}{code:java}
> {"1":"a", "2":"b"}{code}|
> |variant_from_arrays|Transforms two arrays of
> keys and values respectively, into a new variant object.|{code:java}
> SELECT variant_from_arrays(array('1', '2'), array('a', 'b'));
> {code}{code:java}
> {"1":"a", "2":"b"}{code}|
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
