EamesTrinh commented on PR #58831:
URL: https://github.com/apache/spark/pull/58831#issuecomment-5858992270
> > For example:
> > ```sql
> > CREATE TABLE t (s STRUCT<x: INT, y: INT>) USING <v2-provider>;
> >
> > INSERT WITH SCHEMA EVOLUTION INTO t (s)
> > SELECT named_struct('y', 1, 'x', 2, 'z', 3);
> > ```
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > The first two input fields should map positionally to `x` and `y`, while
`z` should be added through schema evolution.
>
> Why? IMHO this looks like WTF moment for a user.
@dolfinus The key consideration here is consistency with existing
column-list insert semantics.
Today, without schema evolution:
```sql
CREATE TABLE t (s STRUCT<x: INT, y: INT>) USING <v2-provider>;
INSERT INTO t (s)
SELECT named_struct('y', 1, 'x', 2);
```
This produces `{x: 1, y: 2}` because the nested fields are resolved
positionally.
However, merely adding a trailing field with schema evolution:
```sql
INSERT WITH SCHEMA EVOLUTION INTO t (s)
SELECT named_struct('y', 1, 'x', 2, 'z', 3);
```
currently produces `{x: 2, y: 1, z: 3}`. The presence of the new field
unexpectedly changes how the existing `x` and `y` fields are resolved.
This PR preserves the existing positional semantics, producing `{x: 1, y: 2,
z: 3}`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
