AndreKurait commented on PR #16746: URL: https://github.com/apache/lucene/pull/16746#issuecomment-6591019788
@dweiss ran through test cases across all 3 versions (mainline, this pr, and your linked aircompressor decompressor). I've incorporated a fix to make sure this PR improves performance in every way. > I tested your actual decoder across all four CPU families. On AMD, it’s about 1.75× faster than mainline on logs and 1.33× on XML, but roughly 2.1× slower on 273/274-byte matches; Intel has smaller regressions there, while Graviton improves. The exact cause remains unresolved. My latest revision removes the earlier AMD log/XML regressions while retaining roughly 50× throughput on long distance-three repeats. Your common-copy path looks promising for a separate adaptation. ```markdown > **AI-assisted benchmark summary** > > I tested Lucene mainline (`1c2072f`), this PR (`15ff28d`), and Dawid’s actual `master-jdk25` Aircompressor decoder (`e5c03f0`), compiled from its source rather than a modeled implementation. > > **Method:** Fresh JMH runs on six EC2 instances per CPU family: AMD c8a, Intel m8i (SMT disabled), Graviton4 c8g, and Graviton5 m9g. Each instance tested all three implementations on identical inputs across 16 categories, with decoder order balanced across rounds. > > All workers used Corretto 25.0.5, one benchmark thread pinned to a CPU, a fixed 512 MiB G1 heap, and normal tiered/C2, SuperWord, escape-analysis, and range-check optimizations enabled. Each timing fork used 10 × 1-second warmup iterations followed by 5 × 500-ms measurements; the period-4 case used 30 seconds of warmup and 5 × 1-second measurements. > > There were **1,152 timing forks plus 72 separate JIT/PMU diagnostic forks**. Full decoded-byte correctness was checked before timing. Relevant C2 compilations completed before measurement in all diagnostic forks. All timing observations were retained; using medians or the final two measurements changed aggregate speedups by at most 0.51%. > > **Results:** Each cell is **PR / Aircompressor speedup relative to mainline**. Mainline is 1.00×; higher is faster. Values are geometric means across six paired instance rounds. > > | Workload | AMD | Intel | Graviton4 | Graviton5 | > |---|---:|---:|---:|---:| > | Logs, 32 blocks | 1.01 / 1.76 | 0.97 / 1.03 | 0.99 / 1.23 | 0.97 / 1.23 | > | XML | 1.00 / 1.34 | 1.02 / 1.10 | 1.01 / 1.06 | 1.01 / 1.06 | > | Lucene changes | 1.01 / 2.31 | 1.01 / 1.56 | 1.00 / 1.50 | 0.98 / 1.57 | > | Dickens text | 1.00 / 2.24 | 1.00 / 1.68 | 1.00 / 1.54 | 0.98 / 1.50 | > | Geonames NDJSON | 1.02 / 1.43 | 1.00 / 1.15 | 1.02 / 1.20 | 1.02 / 1.19 | > | Logs, 256 blocks | 1.00 / 1.27 | 0.99 / 1.02 | 1.00 / 1.02 | 1.00 / 1.04 | > | Random, 32 blocks | 1.02 / 0.99 | 1.02 / 0.99 | 1.00 / 1.10 | 0.99 / 1.27 | > | Random, 256 blocks | 1.00 / 0.99 | 1.00 / 1.00 | 1.00 / 1.13 | 1.00 / 1.15 | > | 1-byte repeats, 80 KiB | 174.21 / 54.20 | 66.21 / 11.00 | 83.82 / 7.70 | 88.01 / 8.21 | > | 3-byte repeats, 80 KiB | 48.84 / 1.45 | 21.27 / 1.07 | 20.29 / 1.29 | 22.34 / 1.28 | > | Repeat corpus, 16 KiB | 46.05 / 1.51 | 17.27 / 1.11 | 17.76 / 1.29 | 18.19 / 1.27 | > | 18-byte match | 0.99 / 0.97 | 1.03 / 1.17 | 1.01 / 1.05 | 1.02 / 1.07 | > | 273-byte match | 1.01 / 0.48 | 0.99 / 0.97 | 1.00 / 1.18 | 1.00 / 1.18 | > | 274-byte match | 1.38 / 0.46 | 3.42 / 0.98 | 3.68 / 1.21 | 4.02 / 1.20 | > | Short matches, period 2 | 1.01 / 3.75 | 1.00 / 3.28 | 1.00 / 1.96 | 1.00 / 1.98 | > | Short matches, period 4 | 1.01 / 3.58 | 1.00 / 1.96 | 0.99 / 1.18 | 0.99 / 1.19 | > > This PR’s largest gains are on long overlapping repeats across all four CPU families. Ordinary inputs remain much closer to mainline, with some smaller regressions: Intel logs took **3.00% longer** (pointwise 95% interval: 2.26–3.75%), and Graviton5’s changes fixture took **1.62% longer** (0.19–3.08%). The Graviton5 log slowdown was inconclusive. Intervals were calculated from paired log ratios without multiple-comparison adjustment. > > Dawid’s decoder shows promising gains on ordinary inputs and short matches, but its AMD 273/274-byte cases took **110–115% longer** than Lucene mainline. Bounded VarHandle copies for short matches/literals look worth investigating separately, while preserving Lucene’s API and inlining behavior. > > **Scope:** These are full-block decoder microbenchmarks with preallocated outputs and normalized inputs accepted by all three implementations. They do not measure whole-query performance or establish customer workload prevalence. Aircompressor’s dictionary/partial-read contract differs from Lucene’s, so these results do not establish it as a drop-in replacement. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
