[
https://issues.apache.org/jira/browse/SPARK-55689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Johan Lasperas updated SPARK-55689:
-----------------------------------
Description:
Allow the schema of a table to be updated during INSERT operations - before the
data is written - when the (DSv2) table reports capability
`AUTOMATIC_SCHEMA_EVOLUTION`.
Initially allow new columns and nested struct fields present in the source
query to be added to the target table when these don't exist in the target
schema.
This follows up on similar work to add support for schema evolution in MERGE
INTO: https://issues.apache.org/jira/browse/SPARK-54274
was:
Allow the schema of a table to be updated during INSERT operations - before the
data is written - when the (DSv2) table reports capability
`AUTOMATIC_SCHEMA_EVOLUTION`.
Initially allow new columns and nested struct fields present in the source
query to be added to the target table when these don't exist in the target
schema.
This follows up on similar work to add support for schema evolution in MERGE
INTO.
> Support schema evolution in DSv2 INSERTs
> ----------------------------------------
>
> Key: SPARK-55689
> URL: https://issues.apache.org/jira/browse/SPARK-55689
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.2.0
> Reporter: Johan Lasperas
> Priority: Major
>
> Allow the schema of a table to be updated during INSERT operations - before
> the data is written - when the (DSv2) table reports capability
> `AUTOMATIC_SCHEMA_EVOLUTION`.
>
> Initially allow new columns and nested struct fields present in the source
> query to be added to the target table when these don't exist in the target
> schema.
> This follows up on similar work to add support for schema evolution in MERGE
> INTO: https://issues.apache.org/jira/browse/SPARK-54274
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
