nzw921rx opened a new pull request, #12089: URL: https://github.com/apache/seatunnel/pull/12089
## Purpose Improve the JMH comparison report so reviewers can evaluate measurement quality together with the score change. ## Changes - expose Baseline/Candidate Score, CV, and Error values in the comparison table - add relative changes for CV and Error - keep Score Change direction-adjusted so positive is favorable - use compact `B`/`C` labels for Baseline/Candidate in the table - keep the real baseline/candidate refs and commits in the report summary - add regression-report tests ## Scope This is reporting-only. It does not change JMH execution, benchmark inputs, normalized result collection, JSON schema, or the ABBA comparison workflow. ## SeaTunnel benchmark comparison examples: - Baseline: `dev` at `98182ca59c93c10390717af4a49b8806f5c555f1` (2 runs) - Candidate: `PR #12081` at `fa12fbd60c73a8d3d5458eae95abd356812207f9` (2 runs) - Suite: `checkpoint` - Java: `1.8.0_...` - Runner image: `ubuntu-24.04` - CPU: `AMD EPYC 9V74 80-Core Processor` > Baseline and candidate ran alternately on the same worker. Score and pipeline changes are direction-adjusted so positive is favorable; CV and Error changes are relative changes. This observational report does not enforce a regression threshold. ### JMH comparison `B` = Baseline, `C` = Candidate. | Benchmark | Parameters | Score B | Score C | Score Change | CV B | CV C | CV Change | Error B | Error C | Error Change | Unit | | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | --- | | `CheckpointStorage.checkpointIdAtomicIncrement` | `default` | 117.727 | 115.894 | +1.56% | 10.25% | 13.00% | +26.83% | 39.47% | 50.08% | +26.88% | us/op | | `CheckpointStorage.checkpointOverviewIncrementalUpdate` | `default` | 348.524 | 335.939 | +3.61% | 25.94% | 19.55% | -24.63% | 99.87% | 75.29% | -24.61% | us/op | | `CheckpointStorage.checkpointPersistenceTransaction` | `default` | 1,961.800 | 1,991.395 | -1.51% | 5.00% | 7.06% | +41.20% | 19.25% | 27.18% | +41.19% | us/op | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
