pnowojski commented on pull request #32: URL: https://github.com/apache/flink-benchmarks/pull/32#issuecomment-928949800
> Unforunately it's not that perfect, as the ALIGNED checkpoint do not match the math anymore. Or do they? You can run the benchmark with different debloating target to check how it's behaving, but 878ms vs 600ms from the debloating target is not that far off. Maybe a couple of ms are wasted on the initial RPCs/CheckpointCoordinator/sync/async checkpoint phases. Maybe you could check the checkpoint statistics somehow? Furthermore, [there is a 50% threshold for buffer adjustments](https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/config/#taskmanager-network-memory-buffer-debloat-threshold-percentages), so if estimated time to consume the data is 430ms and we would like to reduce it down to 300ms, that's below the threshold value and the new buffer size won't be announced. Having said that, maybe it's a good idea to reduce this threshold to a couple of % for our micro benchmark (more frequent buffer size announcements won't matter much in this case). Maybe it would also help to bring down the noise/`Error` of that benchmark? `± 71.026` is quite a lot, and maybe it's caused by different invocations being stuck on different thresholds? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
