junmuz opened a new pull request, #11017: URL: https://github.com/apache/paimon/pull/11017
### Purpose Prevent out-of-order updates from recreating deleted rows after primary-key compaction prunes delete records. A delete at version `4` continues blocking an insert at version `2`, while an insert at version `5` is accepted. Add the boolean option `delete-ordering-guards.enabled` (**default: `false`**) for fixed-bucket primary-key tables using `deduplicate` or `partial-update`. Guards retain keys and ordering metadata without deleted payloads. Disabling capture preserves existing guards. - Publish guards atomically with compaction, with retry recovery, failure cleanup, and generation fencing across ordering-history resets. - Resolve guards in native snapshot, incremental, streaming, and changelog paths, including partial-update sequence groups and delete before-images. - Consolidate guard segments and protect retained snapshot/tag/branch references during cleanup. Regular consolidation defaults to four segments; full compaction consolidates two or more. Newer inserts do not automatically retire guards. - Preserve guards through native Flink/Spark rescaling and partition expiration, including guard-only partitions. Use native serialization for Spark rescale tasks under Java and Kryo. - Validate ordering-policy changes, bootstrap legacy bucket layouts, and document configuration and maintenance behavior. Scope excludes append/Data Evolution tables, dynamic/postpone buckets, field aggregates, deletion vectors, primary-key indexes and chain tables. TTL expiry creates no guards; overwrite/truncate explicitly resets ordering history. This doesn't impact Iceberg compatibility as it's only considered during write paths. Fixes the issue https://github.com/apache/paimon/issues/5387 ### Tests - **475 core tests across 17 classes passed**, with normal Maven checks and no failures or skips. Coverage includes guard storage, recovery, reads, maintenance, and existing commit behavior. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
