goutamadwant commented on issue #12060: URL: https://github.com/apache/seatunnel/issues/12060#issuecomment-5549588018
@nzw921rx I completed the local comparisons and startup/cadence checks. Sharing the setup and results before proposing a change. ### What I ran - Native ARM Java 11 on macOS, fixed 4 GiB heap, G1 and `ActiveProcessorCount=4`. - Six combinations: `pipelineCount=1/10/100` and `storedDagCount=0/100`. - Each store measurement performs 100 sequential production `IMap.put` calls, reported as microseconds per DAG. - Initial comparison: 48 fresh JVM forks with 10 warmup and 10 measured batches, plus a separate 24-fork comparison using the current 3-warmup/5-measurement settings. - Readiness comparison: another 48 forks covering original/candidate fixtures with and without the same coordinator startup guard, across two reversed-order blocks. - Separate cadence and JFR probes, plus CPU, allocation, GC, wall-time and lock profiling. Profiled scores were not pooled with unprofiled timings. The three valid JMH matrices contain 120 forks and 84,000 measured writes. Results remain separate because the protocols differ. An earlier readiness experiment had artifact-packaging defects and was excluded entirely before repeating it with corrected, verified binaries. ### What the measurements show The current store fixture replays the full WAL three times between batches. The candidate replays once and validates all 100 DAGs. In the separate 100-pipeline cleanup probe: - Median verification/cleanup time: **2,025.11 ms → 60.16 ms**. - Full WAL reads per cleanup: **3 → 1**. - Total cleanup WAL-read bytes across 12 batches: **450,816,984 → 150,272,328**. - WAL bytes appended per write batch were unchanged. This is cheaper untimed fixture work, not a production write speedup. Removing that work also moves subsequent batches earlier in startup and JVM warmup. With the startup guard applied to both fixtures: - Candidate mean latency was over 5% higher in **9 of 12** parameter/block comparisons. - Absolute standard deviation improved in **4 of 12** comparisons; CV also improved in **4 of 12**. - None of the six parameter combinations met the predeclared mean/dispersion screen in both blocks. These are descriptive local results, not statistical non-regression bounds. The guard observes coordinator activation, not completion of all background startup or JVM warmup. ### Validation and next step All 38 benchmark-module tests passed on Java 8 and Java 11, along with all 12 packaged store/load smoke cases. No production storage behavior changed. I don’t think this establishes a variance fix for #12060. Could we agree whether the benchmark should measure startup-sensitive writes or steady-state storage, and which background services should remain active? We can then validate an agreed protocol on Linux matching CI, rather than adding arbitrary sleeps or tuning warmup until CV looks better. The local filesystem read-back checks establish visible WAL contents, not real HDFS sync latency or crash durability. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
