pnowojski commented on issue #11088: [FLINK-16012][runtime] Reduce the default number of buffers per channel from 2 to 1 URL: https://github.com/apache/flink/pull/11088#issuecomment-592564275 Yes, thank you for the results @wsry. Indeed it looks like idea of decreasing the exclusive buffers on the receiver side by default to 1 is worth pursuing. Actually I'm a bit surprised that reducing exclusive buffers to 1 on the sender side is causing problems. The number of 2 exclusive buffers + ~8 floating was estimated to provide good bandwidth for single channel performance. Assuming 1ms round trip for requesting/receiving a network credit 10 buffers alone should support ~320MB/s network traffic. (32KB*10/1ms = 320MB/s). What is the network traffic of our network benchmarks? 28161 ops/ms * 32 bytes/ops = ~900MB/s. From this, I would expect 30 floating buffers to suffice on your machine, unless the round trip of credit assignment on local host is more than 1ms? Regardless of that, as I discussed with @zhijiangW offline. For many scenarios (probably most) we can accept a performance regression in favour of smaller memory requirements and faster checkpointing under back pressure - at least as an alternative configuration of Flink. In other words, we could document two recommended configuration setups: 1. (current one) hight throughput mode 2. (limited exclusive buffers, as we are discussing, maybe down to 0) fast checkpointing/low in-flight data mode.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
