SteNicholas opened a new issue, #1424:
URL: https://github.com/apache/paimon-rust/issues/1424

   ### Search before asking
   
   - [x] I searched existing issues and pull requests. No open issue or active 
PR tracking this specific SQL UPDATE gap was found on November 17, 2026.
   
   ### Motivation
   
   DataFusion SQL still rejects UPDATE on ordinary primary-key tables, although 
the core writer supports upserts and SQL DELETE on deduplicate primary-key 
tables is now available through #1392.
   
   For example, with a Paimon catalog configured:
   
   ```sql
   CREATE TABLE paimon.test_db.pk_update_example (
       id INT NOT NULL,
       v INT,
       PRIMARY KEY (id)
   ) WITH ('bucket' = '1', 'merge-engine' = 'deduplicate');
   
   INSERT INTO paimon.test_db.pk_update_example VALUES (1, 10), (2, 20);
   UPDATE paimon.test_db.pk_update_example SET v = v + 1 WHERE id = 1;
   ```
   
   The UPDATE currently fails with `UPDATE on primary-key tables without 
data-evolution is not supported`. It should leave `(1, 11)` and `(2, 20)` for 
the proposed supported configuration.
   
   The Rust rejection is in 
[execute_update](https://github.com/apache/paimon-rust/blob/1920eab1c60d2b5680fb767dbd60ae0cf668db78/crates/integrations/datafusion/src/update.rs#L60-L70).
 Java/Spark's 
[performUpdateForPkTable](https://github.com/apache/paimon/blob/00b4d05a6cc118091a38eeca23549519cfb775f4/paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/commands/UpdatePaimonTableCommand.scala#L61-L67)
 projects the updated rows, attaches UPDATE_AFTER, and uses the normal writer.
   
   ### Solution
   
   Start with ordinary primary-key tables using `merge-engine=deduplicate`, 
reusing the existing write/commit pipeline and the applicable safeguards from 
primary-key DELETE.
   
   - Read matching logical rows from one captured snapshot, evaluate SET 
expressions against the original row, and write complete updated rows with 
UPDATE_AFTER. Preserve fields not assigned by SET and reuse existing Arrow 
batch normalization.
   - Reject assignments to primary-key or partition columns in the first 
version. Reject other merge engines and unsupported configurations such as 
`rowkind.field` or query-authorized targets rather than silently broadening 
their semantics.
   - Preserve configured sequence values unless explicitly assigned, and retain 
the configured sort order. Define how an assigned sequence affects the 
resulting upsert; an UPDATE must not be assumed to override a higher-priority 
concurrent version.
   - Define snapshot and concurrent-upsert behavior, including self-references 
in predicates. Return the matched logical-row count; a statement matching no 
rows should publish no snapshot.
   - Keep publication retries on the prepared commit messages, without 
rerunning the query. Re-executing SQL is a new operation against a new read 
view. Discard uncommitted staging output on scan/write failures, but preserve 
files whose publication outcome is uncertain.
   - Preserve existing target-resolution restrictions for temporary objects, 
branches and time-travel options. Validate the resolved target before obtaining 
persistent keys. Unsupported UPDATE clauses must be rejected explicitly.
   
   Suggested validation:
   
   - Predicate and full-table UPDATE, no matches, SQL NULL behavior, multiple 
assignments using the original values, and unchanged columns.
   - Partitioned tables, composite primary keys, multiple stored versions, 
quoted identifiers and string-view inputs.
   - Rejected primary-key/partition assignments and unsupported merge 
engines/configurations.
   - Sequence fields with ascending/descending order, explicit sequence 
assignments, deterministic concurrent upserts, and commit retries without a 
second scan.
   - Target shadowing/replacement and staging cleanup after partial output or a 
failed scan.
   
   ### Anything else?
   
   Related: #1379 / #1392 for deduplicate primary-key SQL DELETE.
   
   SQL MERGE INTO, primary-key/partition-key changes, additional merge engines 
and changes to the existing append-table or Data Evolution UPDATE 
implementations are outside this first change.
   
   Source baseline: paimon-rust `1920eab1c60d2b5680fb767dbd60ae0cf668db78`, 
compared with Java Paimon `00b4d05a6cc118091a38eeca23549519cfb775f4`. The gap 
was checked in source; this request does not claim a new runtime reproduction.
   
   ### Willingness to contribute
   
   - [ ] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to