[
https://issues.apache.org/jira/browse/SPARK-59793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59793:
-----------------------------------
Labels: pull-request-available (was: )
> Preserve column default expressions in CTAS and RTAS
> ----------------------------------------------------
>
> Key: SPARK-59793
> URL: https://issues.apache.org/jira/browse/SPARK-59793
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Szehon Ho
> Priority: Major
> Labels: pull-request-available
>
> CTAS and RTAS inherit column default metadata from their input query, but do
> not pass the available current-default connector expression to a Data Source
> V2 catalog. CREATE TABLE and REPLACE TABLE provide both SQL and a V2
> expression when translation is supported.
> For example:
> {code:sql}
> CREATE TABLE catalog.src (i INT DEFAULT 42) USING provider;
> CREATE TABLE catalog.dst USING provider AS SELECT * FROM catalog.src;
> {code}
> The default passed to the catalog for dst has SQL "42" and an
> existence/backfill literal of 42, but getExpression() is null. This also
> affects REPLACE TABLE AS SELECT and CREATE OR REPLACE TABLE AS SELECT,
> including staging catalogs.
> The proposed fix is scoped to the common CTAS/RTAS execution helper. It
> translates the current-default expression already carried in the query schema
> and passes it alongside the SQL text. It preserves the separate existence
> default and retains SQL fallback when no translatable expression is
> available. The shared schema-to-V2-column conversion is unchanged.
> Regression coverage includes literal, arithmetic/cast, and scalar-function
> defaults with expression folding both enabled and disabled, plus distinct
> existence defaults and SQL fallback.
> Pull request: https://github.com/apache/spark/pull/59066
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
