CryoThrust commented on issue #12086: URL: https://github.com/apache/seatunnel/issues/12086#issuecomment-5537141803
For the benchmark umbrella, one useful review gate could be a small evidence bundle attached to each optimization PR: - exact benchmark selector, parameters, JDK/JVM flags, runner class, and storage configuration; - baseline and candidate JMH JSON from the same runner class, with mean, error, CV, p95/p99 where applicable, allocation rate, and GC time; - a profiler artifact that identifies the production call chain (CPU/wall/lock/GC/JFR), not only a changed source line; - a correctness check covering state durability, ordering, and failure/restart behavior for the optimized path; - an explanation of any benchmark fixture change separately from production-code changes. For noisy Hazelcast/IMap benchmarks, I would also record warmup/measurement iteration counts and report both median CV and the full per-iteration distribution. A statistically faster mean with materially worse tail latency or variance should not be treated as an unconditional improvement. A lightweight machine-readable manifest for these fields would make the diagnostics workflow and later result comparison much easier to automate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
