nzw921rx opened a new issue, #12063: URL: https://github.com/apache/seatunnel/issues/12063
## Description This is a focused investigation and improvement task for the latency variance observed in the IMap job-lifecycle growth benchmarks. It is not a performance regression report. The benchmark results show noticeable sample-to-sample variance in both running-job growth and completed-job history growth: | Benchmark | Parameters | Score | Error | CV | | --- | --- | ---: | ---: | ---: | | `runningJobGrowth` | `initialStoredJobCount=0` | 410.975 us/op | 15.74% | 14.72% | | `runningJobGrowth` | `initialStoredJobCount=1000` | 342.508 us/op | 10.58% | 9.90% | | `completedJobHistoryGrowth` | `initialStoredJobCount=0` | 2,219.33 us/op | 10.63% | 9.94% | | `completedJobHistoryGrowth` | `initialStoredJobCount=1000` | 2,015.11 us/op | 10.03% | 9.38% | Benchmark run: https://github.com/apache/seatunnel/actions/runs/33750118527 Other benchmarks executed in the same environment produced substantially lower variance. These two methods also share the same job-lifecycle growth workload, so they should be investigated together. The goals of this issue are to: 1. Identify whether the variance comes from the benchmark fixture, Hazelcast IMap operations, serialization/allocation, WAL/FileMapStore persistence, lifecycle operations, or the execution environment. 2. Improve the benchmark fixture or methodology if it causes the instability. 3. Optimize the production job-lifecycle storage path if profiling confirms an implementation bottleneck. 4. Preserve running-job and completed-job state correctness and durability. ## Relevant benchmarks ```text IMapJobStorageBenchmark.runningJobGrowth IMapJobStorageBenchmark.completedJobHistoryGrowth ``` ## Profiling tools For an overview of the Zeta benchmark suite and usage, see the [SeaTunnel Zeta Benchmark Guide](https://seatunnel.apache.org/docs/engines/zeta/benchmark). SeaTunnel provides the `Benchmarks Diagnostics` workflow for running one exact benchmark method with CPU, wall-clock, lock, GC, or JFR profiling: https://github.com/apache/seatunnel/actions/workflows/benchmarks_diagnostics.yml The benchmark can also be run directly from the GitHub Actions page by selecting **Benchmarks Diagnostics**, clicking **Run workflow**, and providing the target branch, tag, commit SHA, or trusted PR number together with the exact benchmark method. ## Local profiling Build the benchmark module: ```bash ./mvnw -Pbenchmark -pl seatunnel-benchmarks -am -DskipTests package ``` Run a profiler locally: ```bash bash tools/benchmarks/profile_benchmarks.sh profile cpu \ --repository . \ --benchmark 'IMapJobStorageBenchmark.runningJobGrowth$' ``` Replace `cpu` with `wall`, `lock`, or `gc` as needed. The same command can be used for: ```text IMapJobStorageBenchmark.completedJobHistoryGrowth$ ``` Capture a JFR recording: ```bash bash tools/benchmarks/profile_benchmarks.sh capture jfr \ --repository . \ --benchmark 'IMapJobStorageBenchmark.runningJobGrowth$' ``` CPU, wall-clock, and lock profiling require `ASYNC_PROFILER_HOME`. The GitHub Actions workflow installs the required profiler automatically. ## Expected outcome This issue should result in both root-cause analysis and a focused improvement: - Identify and explain the primary source of the latency variance. - Improve the benchmark when the fixture or methodology is responsible. - Optimize the production job-lifecycle storage path when an implementation problem is confirmed. - Add or update benchmark coverage to validate the improvement. - Provide before-and-after results collected with the same JDK, runner, benchmark arguments, and storage configuration. - Compare both operation latency and variance before and after the change. - Add appropriate tests and confirm that running-job and completed-job state correctness and durability remain unchanged. The implementation should focus on the confirmed source and avoid unrelated, broad state-store refactoring. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
