SEZ9 commented on issue #12063: URL: https://github.com/apache/seatunnel/issues/12063#issuecomment-5595009727
Thanks @Rangsh for the detailed update, and for following the evidence pattern @nzw921rx suggested. On the `apache/seatunnel` dispatch: I can run `Benchmarks Diagnostics` there against a `dev` baseline using the same methods and `initialStoredJobCount` values as your local run, and I'll post the run links here. Cancelling the queued fork runs makes sense; the fork runs you linked (34229761252 / 34230694806 for `dev`, 34229094919 / 34230337852 for the PR branch) are sufficient as interim evidence. On the interim results: the local before/after (`8bea8c681` vs `991e67b40`) together with the wall-stack frame counts (`FileMapStore.loadAll` 4 → 1, `WALReader.loadAllData` 5 → 1, reload now only under `verifyLastGrowthPhaseDurability`) are consistent with the variance coming from the growth benchmark fixture rather than from IMap itself, which is the kind of complete causal chain that was asked for. Two follow-ups before I treat this point as closed: 1. Do the `completedJobHistoryGrowth` rows and the GC diagnostics show the same shift for the `initialStoredJobCount=1000` case, i.e. reload cost only at trial tear-down? 2. In the local run, the `runningJobGrowth` / `initialStoredJobCount=0` CV is higher after (31.68%) than before (20.89%), while the 1000 case improved a lot. Once the same-runner run is in, let's check whether that spread persists or was local noise. With those answered, I'm comfortable treating the fixture-cost explanation as confirmed for this issue. <!-- streview-comment:915 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
