SteNicholas opened a new pull request, #1433:
URL: https://github.com/apache/paimon-rust/pull/1433

   ### Purpose
   
   Linked issue: closes #1424
   
   Enable SQL UPDATE on ordinary primary-key tables using 
`merge-engine=deduplicate`. DataFusion previously rejected these updates even 
though the Paimon writer supports upserts. An update such as `SET v = v + 1 
WHERE id = 1` now updates the matching logical row while preserving unassigned 
fields.
   
   ### Brief change log
   
   - Project complete `UPDATE_AFTER` rows, evaluate all SET expressions against 
the original row, and coerce results to the table schema. Reject primary-key or 
partition assignments and unsupported configurations or UPDATE clauses.
   - Share primary-key DELETE target validation, snapshot planning and writes. 
Pin live self-references in predicates and SET expressions to the captured 
snapshot, merge uncompacted deletion-vector versions, and preserve explicit 
historical or incremental views.
   - Preserve unassigned sequence values and configured sort order. Report 
matched logical-row counts, including updates that lose under sequence 
ordering, and skip commits when no rows match. Concurrent writes retain normal 
upsert semantics.
   - Reuse writer resources, Arrow normalization and staging cleanup. Retry 
publication with prepared messages without rescanning, and retain prepared 
files when publication fails with an uncertain outcome.
   
   ### Tests
   
   Passed locally with Rust 1.98.0: 477 tests across DataFusion unit tests and 
the primary-key UPDATE, primary-key DELETE, existing UPDATE and existing DELETE 
integration suites.
   
   ```text
   cargo +1.98.0 test --locked --offline -p paimon-datafusion --test 
pk_update_tests
   cargo +1.98.0 test --locked --offline -p paimon-datafusion --lib --test 
pk_delete_tests --test update_tests --test delete_tests
   cargo +1.98.0 fmt --all -- --check
   cargo +1.98.0 clippy --locked --offline -p paimon-datafusion --all-targets 
-- -D warnings
   ```
   
   Coverage includes logical-row selection, original-value assignments, NULLs 
and casts, quoted identifiers, partitioned/composite keys, sequence ordering, 
deterministic concurrent upserts, snapshot pinning, publication retries, target 
replacement and staging cleanup. Boundary assertions for unchanged values, 
identifier normalization and file preservation are included in existing tests.
   
   ### API and Format
   
   Adds SQL UPDATE support for deduplicate primary-key tables. No public Rust 
API or storage-format changes. Primary-key/partition-key changes, other merge 
engines and serializable update isolation remain outside this change.
   
   ### Documentation
   
   Updated `docs/src/sql.md` with supported UPDATE semantics, restrictions, 
snapshot/concurrent-write behavior, affected-row counts and publication failure 
handling.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to