leaves12138 opened a new pull request, #9189: URL: https://github.com/apache/paimon/pull/9189
## What changed - Add `_ROW_ID` as the final sort key for sorted global index builds. - Add a unit test that locks the sort-key order to build task ID, index key, and row ID. ## Why Range shuffle groups duplicate sampled boundaries and randomly assigns matching records among their range indexes. When the sort key contains only the build task ID and index key, a hot index key can therefore be spread across writer partitions in an order that produces overlapping output key ranges within the same row-range group. Using `_ROW_ID` as a tie-breaker makes the full shuffle key unique while keeping equal index keys adjacent. The actual index key is unchanged, and neighboring output files may still meet at the same index-key boundary without crossing each other. ## Impact New sorted global index builds preserve ordered file key ranges, preventing downstream BTree compaction from receiving overlapping inputs caused by duplicate range-shuffle boundaries. ## Validation - `mvn -pl paimon-flink/paimon-flink-common -Pflink1 -DwildcardSuites=none -Dtest=SortedIndexTopoBuilderTest test` (8 tests passed) - `mvn -pl paimon-flink/paimon-flink-common -Pflink1 -DskipTests spotless:check` - `mvn -pl paimon-flink/paimon-flink-1.20 -am -Pflink1 -Pfast-build -DskipTests install -T 16` - `git diff --check` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
