Rangsh commented on PR #12081: URL: https://github.com/apache/seatunnel/pull/12081#issuecomment-5537004723
Thanks for the detailed review — I’ve addressed the inline comments and updated the PR description with the evidence pack. ### Inline review feedback 1. **`docs/zh` / `docs/en` CV paragraphs** — Reverted. The variance analysis lives in this PR description instead of permanent benchmark docs. 2. **`CheckpointStorageBenchmark` extra javadoc** — Removed. 3. **`calculateStateSize` stream→loop** — Agreed this is not a meaningful CV driver. Reverted the stream→loop changes (and the related unit test) so this PR stays focused on the WAL sync path. ### Core fix retained - Exactly one durable sync in `HdfsWriter.flush()` - `RequestFuture` completion / success semantics + timed batch wait - `WALWorkHandler` always calls `done()` on APPEND failures ### Evidence chain (before / after) Same suite (`CheckpointStorageBenchmark`), fork Actions `Benchmarks` workflow, local `file:///` MapStore (no remote HDFS). | Role | Commit | Workflow run | | --- | --- | --- | | Before (`dev`) | `98182ca59` | `33840694352` | | After (this PR) | `fa12fbd60` | `33840699925` | **Java 8** | Benchmark | Before Score / Error / CV | After Score / Error / CV | | --- | --- | --- | | `checkpointIdAtomicIncrement` | 160.9 us/op · 15.60% · 14.59% | 111.2 us/op · 10.28% · 9.62% | | `checkpointOverviewIncrementalUpdate` | 381.6 us/op · 16.21% · 15.16% | 320.5 us/op · 15.05% · 14.08% | **Java 11** | Benchmark | Before Score / Error / CV | After Score / Error / CV | | --- | --- | --- | | `checkpointIdAtomicIncrement` | 122.5 us/op · 19.44% · 18.18% | 122.7 us/op · 14.41% · 13.48% | | `checkpointOverviewIncrementalUpdate` | 355.3 us/op · 19.20% · 17.96% | 321.9 us/op · 16.69% · 15.61% | **Causal chain (wall profile, Java 11 Diagnostics):** `FileMapStore.store` → `HdfsWriter.write/flush` → `FSDataOutputStream.hsync` Caveats I’m being explicit about: - GitHub-hosted runners used different CPUs across before/after hosts, so absolute `Score` is observational; the directional Error/CV drop is the main signal for #12058. - Residual CV remains by design (still one durable sync per measured write-through op). Happy to add cpu/lock flame graphs as well if you still want them on top of the wall profiles. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
