FenjuFu opened a new pull request, #12248: URL: https://github.com/apache/seatunnel/pull/12248
### Purpose of this pull request Refs #12086. Preserve JMH fork boundaries in normalized reports and expose the evidence needed to distinguish variation within a fork from shifts between forks. The normalized metric currently flattens rawData. This adds an optional fork_samples field and a collapsible table with sample counts, per-fork means/CVs and the CV of unweighted fork means. ABBA diagnostics remain separate for every baseline/candidate run. Existing Score, Error, overall CV and comparison formulas are unchanged. ### Does this PR introduce _any_ user-facing change? Benchmark reports gain fork diagnostics when grouped samples are available. Old schema-version-1 reports still render without inventing missing grouping. CV is unavailable for insufficient observations or a zero mean; non-finite observations are not treated as evidence of stability. English and Chinese benchmark guides explain the fields and limits. This is a reporting improvement on dev. It does not modify engine behavior or claim a production speedup or root cause. ### How was this patch tested? - 14 focused normalizer/reporter tests passed, including equal aggregate distributions with different fork structure, unequal fork lengths, missing/zero/non-finite observations, legacy reports and ABBA run separation. - Replayed the [Java 11 artifact from run 33750118527](https://github.com/apache/seatunnel/actions/runs/33750118527): all 11 normalized JMH metrics retain every prior field exactly, while each now preserves its three groups of five samples. - For runningJobGrowth with initialStoredJobCount=0, the existing overall CV remains 14.72%. New within-fork CVs are 11.31%, 11.09%, 20.90%, and fork-mean CV is 4.04%. These are descriptive results from the existing artifact, not a fresh benchmark or causal conclusion. - Python syntax/undefined-name checks and git diff --check passed. - The full 35-test script suite was attempted locally: 7 unchanged profiling tests fail on Windows path conventions; affected normalizer/reporter tests pass. Linux CI is pending. Local whole-repository Spotless was stopped while traversing unrelated Java sources under heavy disk load; no successful local Maven verification is claimed. Reproduce the focused tests: ```bash python3 -m unittest discover -s tools/benchmarks -p 'test_save_jmh_result.py' python3 -m unittest discover -s tools/benchmarks -p 'test_regression_report.py' ``` ### Check list - [x] New behavior has focused regression coverage. - [x] Updated English and Chinese documentation. - [x] No new dependency, JAR, connector or incompatible format change. - [x] Commit includes a matching Signed-off-by trailer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
