XiaoHongbo-Hope opened a new pull request, #9108: URL: https://github.com/apache/paimon/pull/9108
### Purpose Manifest sort compaction currently uses QuickSort for every in-memory external-sort run. This follows the radix-sort optimization in #8904 and applies it to the generated normalized keys used by manifest sorting. ### Changes - Add an in-place MSD radix sorter for generated normalized keys. - Fall back to QuickSort for small ranges and for equal partial normalized keys. - Use the selected sorter for both in-memory output and spilled runs. - Keep QuickSort as the default for all existing sort-buffer callers; only manifest external sort opts into radix sort. The in-place implementation uses bounded auxiliary memory (at most two 256-entry arrays per normalized-key byte), so it preserves the external sort buffer's memory bound. ### Benchmark `ManifestFileSorterBenchmark`, 48 manifests x 10,000 entries, 64 MiB sort buffer, one warmup and two measured iterations: | Case | QuickSort best | Radix best | QuickSort allocation | Radix allocation | | --- | ---: | ---: | ---: | ---: | | Full, 480k entries | 2687.7 ms | 2636.5 ms | 2243.2 MiB | 2228.8 MiB | | Minor with deletes, 660k entries | 4081.5 ms | 3979.9 ms | 3033.9 MiB | 2998.5 MiB | This is an end-to-end benchmark including manifest read and rewrite I/O. ### Tests - `NormalizedKeyRadixSortTest` - `BinaryExternalSortBufferTest` - `ManifestFileMetaTest` - Spotless and Checkstyle -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
