Rangsh commented on PR #12172: URL: https://github.com/apache/seatunnel/pull/12172#issuecomment-5577568892
@nzw921rx The fork Ubuntu-hosted ABBA run finished successfully. - Run: https://github.com/Rangsh/seatunnel/actions/runs/34130412395 - In-fork PR (same commit as this PR): https://github.com/Rangsh/seatunnel/pull/1 - Sequence: `baseline -> PR -> PR -> baseline` - Baseline: `72acda570` (`dev`) - Candidate: `f28464c21` (this PR) - Suite: `DebeziumJsonFormatBenchmark` ### Java 11 (Temurin 11.0.32, GitHub-hosted Ubuntu) | Benchmark | Score B | Score C | Score Change | CV B | CV C | CV Change | Error B | Error C | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | `deserializeInsertEvent` | 331.370 | 473.780 | **+42.98%** | 0.51% | 0.25% | **-51.42%** | 1.97% | 0.96% | | `deserializeUpdateEvent` | 230.202 | 336.663 | **+46.25%** | 0.57% | 0.21% | **-62.74%** | 2.19% | 0.82% | | `serializeInsertEvent` | 477.439 | 553.647 | **+15.96%** | 0.52% | 0.86% | +63.16% | 2.02% | 3.29% | | `serializeMergedUpdateEvent` | 304.252 | 360.033 | **+18.33%** | 0.32% | 0.29% | **-11.09%** | 1.25% | 1.11% | ### Java 8 (Temurin 1.8.0_504, GitHub-hosted Ubuntu) | Benchmark | Score B | Score C | Score Change | CV B | CV C | CV Change | Error B | Error C | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | `deserializeInsertEvent` | 369.937 | 527.528 | **+42.60%** | 0.05% | 0.07% | +35.13% | 0.19% | 0.25% | | `deserializeUpdateEvent` | 261.869 | 363.632 | **+38.86%** | 0.08% | 0.11% | +32.23% | 0.33% | 0.43% | | `serializeInsertEvent` | 451.028 | 629.284 | **+39.52%** | 0.07% | 0.11% | +44.45% | 0.28% | 0.41% | | `serializeMergedUpdateEvent` | 271.464 | 394.033 | **+45.15%** | 0.09% | 0.08% | **-6.54%** | 0.33% | 0.31% | ### Reading of the earlier Java 11 CV / Error concern On this same-worker Ubuntu ABBA: 1. Throughput improved for all four methods on both JDKs. 2. On Java 11, the earlier “CV/Error up across all four methods” trend does **not** reproduce: - deserialize methods are more stable on the candidate (CV / Error down) - `serializeMergedUpdateEvent` is slightly more stable - only `serializeInsertEvent` has a higher CV / Error, and the absolute CV is still low (`0.86%`) 3. On Java 8, some relative CV/Error percentages rise, but absolute values remain very low (`≈0.07%–0.11%`), so this looks like normal low-noise variation rather than a stability regression. Overall, these fork-hosted ABBA results support the throughput win and do not show a consistent Java 11 stability regression introduced by this change. Happy to add a focused Java 11 GC/CPU diagnostic for `serializeInsertEvent` if you still want that extra check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
