pnowojski commented on issue #11088: [FLINK-16012][runtime] Reduce the default number of buffers per channel from 2 to 1 URL: https://github.com/apache/flink/pull/11088#issuecomment-587424076 Keep in mind that the purpose of this is to reduce the amount of in-flight data. So just moving the exclusive buffers to floating pool, is not helping us with this. I was hoping that in setups with 1000 channels, if we decrease the exclusive buffers from 2000 to 1000, while increasing floating from 7 to for example 30, might give us good balance. > maybe the only problem is that the floating buffer can't be always fulfilled That's also true for exclusive buffers. If there is not enough memory/buffers on the TM, network stack will allocate only one exclusive buffer per channel. That also means, for some users (I would guess for quite a lot of them) decreasing the exclusive buffers from 2 to 1, will not reduce the amount of in-flight records.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
