pjfanning commented on PR #3737: URL: https://github.com/apache/pekko/pull/3737#issuecomment-6520687473
Follow-up JMH run with more forks and interleaved variants, addressing the caveats in my previous comment. **Setup.** Both variants are in one JMH invocation through a temporary (uncommitted) `@Param` in `TlsBenchmark` that picks the TLS attributes at runtime, so nothing is rebuilt between them: - `plain` = this PR (only plaintext input unbatched) - `both` = `main` (ciphertext input also unbatched) 4 rounds of `bench-jmh/Jmh/run -wi 5 -i 10 -w 1s -r 1s -f1 -t1 -p implementation=graphstage .*TlsBenchmark.warmRoundTrip`, alternating which variant runs first (rounds 1 and 3: `plain,both`; rounds 2 and 4: `both,plain`; the order was confirmed in the JMH output). That is 4 forks × 10 measured iterations per variant and payload. JDK 17, macOS laptop. Score per round (ops/ms, higher is better): | payload | variant | round 1 | round 2 | round 3 | round 4 | mean | |---|---|---|---|---|---|---| | 256 B | this PR | 92.2 | 138.8 | 125.1 | 137.4 | 123.4 | | 256 B | main | 121.5 | 112.2 | 138.6 | 131.9 | 126.0 | | 4 KiB | this PR | 34.3 | 49.8 | 47.5 | 48.9 | 45.1 | | 4 KiB | main | 41.2 | 50.3 | 50.7 | 51.8 | 48.5 | | 64 KiB | this PR | 4.97 | 4.81 | 4.79 | 4.80 | 4.84 | | 64 KiB | main | 4.58 | 4.92 | 4.85 | 5.01 | 4.84 | Difference within each round (this PR vs main): | payload | round 1 | round 2 | round 3 | round 4 | mean | |---|---|---|---|---|---| | 256 B | −24.1% | +23.8% | −9.8% | +4.2% | −1.5% | | 4 KiB | −16.8% | −0.9% | −6.4% | −5.7% | −7.5% | | 64 KiB | +8.5% | −2.2% | −1.2% | −4.2% | +0.2% | **Reading:** - **256 B:** no signal. The sign follows run order (whichever variant ran second won in every round), so position effects dominate. - **4 KiB:** this PR is slower in all 4 rounds, whichever variant ran first. Round 1 looks cold (both variants low); without it the gap is about −4%. A small, consistent regression, not noise. - **64 KiB:** flat. So on this echo/round-trip benchmark, batching ciphertext input does not help, and at 4 KiB it costs a few percent. A plausible cause is that in a ping-pong exchange, batches are usually one element, so batching adds bookkeeping without saving messages. This benchmark does not model bulk inbound transfer, where batching could still pay off, but there is no evidence for that yet. I'd not merge this as a performance change on the current evidence. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
