wsry commented on issue #11088: [FLINK-16012][runtime] Reduce the default number of buffers per channel from 2 to 1 URL: https://github.com/apache/flink/pull/11088#issuecomment-593072136 @pnowojski IMO, the reason why reducing buffers of sender side is causing problem is that our benchmark can request a large amount of buffers in a really short time. Our benchmark uses roundrobin partitioner and the size of record is fixed, so as a result, almost all subpartitions will fill up the current buffer and need to request next one. If there are no enough buffers at the time, the sender thread need to wait the all previously sent buffers to be recycled which need some time. I agree that we need to document the configuration setups you mentioned and except for reducing the number of buffers, users can also try to reduce the size of network buffer to reduce in-flight data in cases of back pressure (or maybe we can reduce the size of buffer dynamically).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
