SteNicholas opened a new issue, #1424:
URL: https://github.com/apache/paimon-rust/issues/1424
### Search before asking
- [x] I searched existing issues and pull requests. No open issue or active
PR tracking this specific SQL UPDATE gap was found on November 17, 2026.
### Motivation
DataFusion SQL still rejects UPDATE on ordinary primary-key tables, although
the core writer supports upserts and SQL DELETE on deduplicate primary-key
tables is now available through #1392.
For example, with a Paimon catalog configured:
```sql
CREATE TABLE paimon.test_db.pk_update_example (
id INT NOT NULL,
v INT,
PRIMARY KEY (id)
) WITH ('bucket' = '1', 'merge-engine' = 'deduplicate');
INSERT INTO paimon.test_db.pk_update_example VALUES (1, 10), (2, 20);
UPDATE paimon.test_db.pk_update_example SET v = v + 1 WHERE id = 1;
```
The UPDATE currently fails with `UPDATE on primary-key tables without
data-evolution is not supported`. It should leave `(1, 11)` and `(2, 20)` for
the proposed supported configuration.
The Rust rejection is in
[execute_update](https://github.com/apache/paimon-rust/blob/1920eab1c60d2b5680fb767dbd60ae0cf668db78/crates/integrations/datafusion/src/update.rs#L60-L70).
Java/Spark's
[performUpdateForPkTable](https://github.com/apache/paimon/blob/00b4d05a6cc118091a38eeca23549519cfb775f4/paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/commands/UpdatePaimonTableCommand.scala#L61-L67)
projects the updated rows, attaches UPDATE_AFTER, and uses the normal writer.
### Solution
Start with ordinary primary-key tables using `merge-engine=deduplicate`,
reusing the existing write/commit pipeline and the applicable safeguards from
primary-key DELETE.
- Read matching logical rows from one captured snapshot, evaluate SET
expressions against the original row, and write complete updated rows with
UPDATE_AFTER. Preserve fields not assigned by SET and reuse existing Arrow
batch normalization.
- Reject assignments to primary-key or partition columns in the first
version. Reject other merge engines and unsupported configurations such as
`rowkind.field` or query-authorized targets rather than silently broadening
their semantics.
- Preserve configured sequence values unless explicitly assigned, and retain
the configured sort order. Define how an assigned sequence affects the
resulting upsert; an UPDATE must not be assumed to override a higher-priority
concurrent version.
- Define snapshot and concurrent-upsert behavior, including self-references
in predicates. Return the matched logical-row count; a statement matching no
rows should publish no snapshot.
- Keep publication retries on the prepared commit messages, without
rerunning the query. Re-executing SQL is a new operation against a new read
view. Discard uncommitted staging output on scan/write failures, but preserve
files whose publication outcome is uncertain.
- Preserve existing target-resolution restrictions for temporary objects,
branches and time-travel options. Validate the resolved target before obtaining
persistent keys. Unsupported UPDATE clauses must be rejected explicitly.
Suggested validation:
- Predicate and full-table UPDATE, no matches, SQL NULL behavior, multiple
assignments using the original values, and unchanged columns.
- Partitioned tables, composite primary keys, multiple stored versions,
quoted identifiers and string-view inputs.
- Rejected primary-key/partition assignments and unsupported merge
engines/configurations.
- Sequence fields with ascending/descending order, explicit sequence
assignments, deterministic concurrent upserts, and commit retries without a
second scan.
- Target shadowing/replacement and staging cleanup after partial output or a
failed scan.
### Anything else?
Related: #1379 / #1392 for deduplicate primary-key SQL DELETE.
SQL MERGE INTO, primary-key/partition-key changes, additional merge engines
and changes to the existing append-table or Data Evolution UPDATE
implementations are outside this first change.
Source baseline: paimon-rust `1920eab1c60d2b5680fb767dbd60ae0cf668db78`,
compared with Java Paimon `00b4d05a6cc118091a38eeca23549519cfb775f4`. The gap
was checked in source; this request does not claim a new runtime reproduction.
### Willingness to contribute
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]