Allison Portis created SPARK-41806:
--------------------------------------
Summary: Use AppendData.byName for SQL INSERT INTO by name for
DSV2 and block ambiguous queries with static partitions columns
Key: SPARK-41806
URL: https://issues.apache.org/jira/browse/SPARK-41806
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.4.0
Reporter: Allison Portis
Currently for INSERT INTO by name we reorder the value list and convert it to
INSERT INTO by ordinal. Since DSv2 logical nodes have the isByName flag we
don't need to do this. The current approach is limiting in that
# Users must provide the full list of table columns (this limits the
functionality for features like generated columns see SPARK-41290)
# It allows ambiguous queries such as INSERT OVERWRITE t PARTITION (c='1') (c)
VALUES ('2') where the user provides both the static partition column 'c' and
the column 'c' in the column list. We should check that the static partition
column is not in the column list.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
