TanYuxin-tyx commented on code in PR #21843: URL: https://github.com/apache/flink/pull/21843#discussion_r1100028107
########## docs/content/docs/deployment/memory/network_mem_tuning.md: ########## @@ -105,12 +105,14 @@ The size of the buffer can be configured by setting `taskmanager.memory.segment- ### Input network buffers -Buffers in the input channel are divided into exclusive and floating buffers. Exclusive buffers can be used by only one particular channel. A channel can request additional floating buffers from a buffer pool shared across all channels belonging to the given input gate. The remaining floating buffers are optional and are acquired only if there are enough resources available. +The number of buffers in one pool calculated according to the above formula can be divided into two parts. The part below this configured value `taskmanager.network.memory.read-buffer.required-per-gate.max` is required. The excess part buffers, if any, is optional. A task will fail if the required buffers cannot be obtained in runtime. A task will not fail due to not obtaining optional buffers, but may suffer a performance reduction. If not explicitly configured, the default value is Integer.MAX_VALUE for streaming workloads, and 1000 for batch workloads. + +Generally, `taskmanager.network.memory.read-buffer.required-per-gate.max` need not configure, and the default value can fulfill most scenes. Setting the option to a lower value, at least 1, can avoid the "insufficient number of network buffers" exception as much as possible, but may reduce performance. Setting the option as Integer.MAX_VALUE to disable the required buffer limit, When disabled, more read buffers may be required, which is good for performance, but may lead to more easily throwing insufficient network buffers exceptions. Review Comment: I have added more descriptions about how to tune the config and the influences after changing the option. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
