Serge Rielau created SPARK-61294:
------------------------------------
Summary: OBJECT type support
Key: SPARK-61294
URL: https://issues.apache.org/jira/browse/SPARK-61294
Project: Spark
Issue Type: New Feature
Components: SQL
Affects Versions: 5.0.0
Reporter: Serge Rielau
{{STRUCT}} fields are matched by position. Assignment, comparison, and set
operations ignore field names, so a reordered query or table can write values
into the wrong fields without an error. {{OBJECT}} makes field names part of
the type: assignment, comparison, and set operations require the same names in
the same order.
Two further gaps motivate the type:
* VARIANT objects have named fields but no declared order or schema.
{{to_variant_object}} sorts fields, and {{schema_of_variant}} returns
{{OBJECT<...>}} only as a string that cannot be stored. {{OBJECT}} gives a
typed, storable home to a VARIANT object with a stated signature.
* Snowflake's structured {{OBJECT}} is the common precedent for users moving
semi-structured data. Matching its functions ({{{}object_delete{}}},
{{{}object_pick{}}}, {{{}ADD FIELDS{}}}, {{{}RENAME FIELDS{}}}) lowers
migration cost. The one deliberate rename is {{{}OBJECT_INSERT{}}}, which
becomes {{{}object_append{}}}.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
