nzw921rx opened a new issue, #12061: URL: https://github.com/apache/seatunnel/issues/12061
## Description This is a focused investigation and improvement task for the latency variance observed in IMap job-storage benchmarks. It is not a performance regression report and does not attempt to compare the performance of Java 8 with Java 11. The benchmark results show noticeable sample-to-sample variance in task-group state transitions and large metrics reports: | Benchmark | Java 8 Error / CV | Java 11 Error / CV | | --- | ---: | ---: | | `taskGroupStateTransition`, stored=0 | 10.47% / 9.79% | 11.86% / 11.09% | | `taskGroupStateTransition`, stored=1000 | 32.29% / 30.21% | 19.32% / 18.07% | | `runningMetricsReport`, tasks=1000 | 10.91% / 10.21% | 20.21% / 18.90% | The smaller `runningMetricsReport` cases with 10 and 100 tasks were substantially more stable, which provides a useful control when investigating the 1,000-task case. Benchmark run: https://github.com/apache/seatunnel/actions/runs/33723940255 The goals of this issue are to: 1. Identify whether the variance comes from the benchmark fixture, Hazelcast operations, serialization/allocation, storage pressure, metrics processing, or the execution environment. 2. Improve the benchmark fixture or methodology if it causes the instability. 3. Optimize the production job-storage or metrics-report path if profiling confirms an implementation bottleneck. 4. Preserve state-transition correctness and metrics-report semantics. ## Relevant benchmarks ```text IMapJobStorageBenchmark.taskGroupStateTransition IMapJobStorageBenchmark.runningMetricsReport ``` ## Profiling tools For an overview of the Zeta benchmark suite and usage, see the [SeaTunnel Zeta Benchmark Guide](https://seatunnel.apache.org/docs/engines/zeta/benchmark). SeaTunnel provides the `Benchmarks Diagnostics` workflow for running one exact benchmark method with CPU, wall-clock, lock, GC, or JFR profiling: https://github.com/apache/seatunnel/actions/workflows/benchmarks_diagnostics.yml The benchmark can also be run directly from the GitHub Actions page by selecting **Benchmarks Diagnostics**, clicking **Run workflow**, and providing the target branch, tag, commit SHA, or trusted PR number together with the exact benchmark method. ## Local profiling Build the benchmark module: ```bash ./mvnw -Pbenchmark -pl seatunnel-benchmarks -am -DskipTests package ``` Run a profiler locally: ```bash bash tools/benchmarks/profile_benchmarks.sh profile cpu \ --repository . \ --benchmark 'IMapJobStorageBenchmark.taskGroupStateTransition$' ``` Replace `cpu` with `wall`, `lock`, or `gc` as needed. The same command can be used for: ```text IMapJobStorageBenchmark.runningMetricsReport$ ``` Capture a JFR recording: ```bash bash tools/benchmarks/profile_benchmarks.sh capture jfr \ --repository . \ --benchmark 'IMapJobStorageBenchmark.taskGroupStateTransition$' ``` CPU, wall-clock, and lock profiling require `ASYNC_PROFILER_HOME`. The GitHub Actions workflow installs the required profiler automatically. ## Expected outcome This issue should result in both root-cause analysis and a focused improvement: - Identify and explain the primary source of the latency variance. - Improve the benchmark when the fixture or methodology is responsible. - Optimize the production job-storage or metrics-report path when an implementation problem is confirmed. - Add or update benchmark coverage to validate the improvement. - Provide before-and-after results collected with the same JDK, runner, benchmark arguments, and storage configuration. - Compare both operation latency and variance before and after the change. - Add appropriate tests and confirm that state-transition correctness and metrics-report semantics remain unchanged. The implementation should focus on the confirmed source and avoid unrelated, broad state-store refactoring. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
