Pandas886 opened a new issue, #505:
URL: https://github.com/apache/paimon-rust/issues/505
### Describe the feature
Add **Diff** incremental planning/reading: compare table state at two
snapshots and emit the net changes (insert / update-before / update-after /
delete) when no pre-materialized changelog is available.
### Motivation
Many tables run with `changelog-producer=none`. Diff is the practical way to
still produce an audit/changelog stream for batch incremental jobs.
### Proposed scope (phase 1)
- Diff planning as per-(partition, bucket) before/after pairs
- Fail loud (no silent wrong answers) for:
- non-deduplicate merge engines
- bucket rescale between snapshots
- deletion vectors / nested/decimal types we can't safely compare yet
- Audit-style row kinds for the diff result
Depends on the base incremental scan + AuditLog read path.
### Out of scope
- Full DV-aware diff
- Non-deduplicate merge engines
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
