jianguotian commented on PR #9792: URL: https://github.com/apache/paimon/pull/9792#issuecomment-5657630054
Geely isolated performance validation is complete. ### Setup - Baseline: `62cd0fd9fd` (current master, including #9791), immutable bundle SHA-256 `2d341684db64c778c7f67da603c00d0f86eb031401239d572d45235c5180f0ad`. - Candidate: `665baa8720`, immutable bundle SHA-256 `1d655fe8cba2c8d1683c02a1ab5ade243453522e72e2a547dd945b29cb900048`. - Current PR head `1cdca5e479` only adds test coverage; production code is unchanged from the measured candidate. - Both layouts were generated from the same source snapshot (`2346`, schema `48`, 18,219,967,325 records) on separate temporary branches. - Both compactions used the same partition field (`dt`), 2 GiB rewrite budget, 32 MiB target manifest size, 1 B full-compaction threshold, and merge minimum count 1. The only layout difference was `manifest-sort.bucket-first=true`. - Reads for both layouts used the same candidate runtime. One warm-up per arm was excluded, then 12 paired runs alternated order (`P,B` / `B,P`). ### Results | Metric | Partition-first | Bucket-first | Change | |---|---:|---:|---:| | Statement execution median | 13.0315 s | 5.8440 s | **55.15% lower / 2.23x faster** | | Statement execution p95 | 14.0510 s | 6.4505 s | 54.09% lower | | Planning median | 9.55 s | 2.30 s | **75.92% lower / 4.15x faster** | | Scanned manifests | 14 | 1 | **92.86% lower / 14x fewer** | Bucket-first won all 12/12 paired runs. The median paired speedup was 2.2326x (range 2.0602x–2.4032x). All 24 measured queries had identical scan invariants: snapshot 2347, 12,830 skipped files, 54 resulting files, 25 splits, and 1,000 returned rows at the configured result limit. Correctness was checked without the result limit: both arms returned 4,280 rows and the same duplicate-preserving aggregate fingerprint (`763133377736551553611`). Both branch snapshots retained schema 48 and the source total record count. The temporary branches/tag were deleted after measurement. The customer main table was not altered or compacted. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
