Szehon Ho created SPARK-61806:
---------------------------------

             Summary: Support generated column values in UPDATE and MERGE
                 Key: SPARK-61806
                 URL: https://issues.apache.org/jira/browse/SPARK-61806
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 5.0.0
            Reporter: Szehon Ho


DSv2 tables that advertise SUPPORTS_GENERATED_COLUMNS_ON_WRITE already allow 
Spark to compute generated columns on INSERT. UPDATE and MERGE are currently 
rejected when such a table has generated columns.

Extend the existing support to row-level writes:
* Recompute omitted generated values from the aligned, post-cast assignments 
for each UPDATE or MERGE action.
* Preserve simultaneous SET semantics and validate explicitly supplied 
generated values against the final row.
* Allow MERGE actions to mix generated and explicitly supplied values, 
including star actions.
* Skip generated-column checks for delete delta records.
* Reuse the modeled generation-expression reader and SQL fallback used by 
INSERT and validation.

For example, for a supported table t with columns a, b, and g GENERATED ALWAYS 
AS (a * 2), starting from (a=1, b=3, g=2):

{code:sql}
UPDATE t SET a = b + 1, b = 100;
{code}

The resulting row should be (a=4, b=100, g=8). Currently the statement is 
rejected as an unsupported table operation.

The implementation depends on SPARK-61805 so a failing generated expression in 
an unselected MERGE action does not fail during optimization.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to