This is an automated email from the ASF dual-hosted git repository. pnowojski pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/flink.git
commit e374b7f16825e38d5de4be1cf307b9abbd5aab63 Author: Piotr Nowojski <[email protected]> AuthorDate: Tue Jun 21 14:30:14 2022 +0200 fixup! [FLINK-26762][docs] Document overdraft buffers --- .../docs/deployment/memory/network_mem_tuning.md | 20 ++++++++++++-------- .../docs/deployment/memory/network_mem_tuning.md | 20 ++++++++++++-------- 2 files changed, 24 insertions(+), 16 deletions(-) diff --git a/docs/content.zh/docs/deployment/memory/network_mem_tuning.md b/docs/content.zh/docs/deployment/memory/network_mem_tuning.md index 8adc281e225..39527284814 100644 --- a/docs/content.zh/docs/deployment/memory/network_mem_tuning.md +++ b/docs/content.zh/docs/deployment/memory/network_mem_tuning.md @@ -120,16 +120,20 @@ Flink 有多个本地缓冲区池 —— 每个输出和输入流对应一个。 #### Overdraft buffers -For each output subtask can also request up to `taskmanager.network.memory.max-overdraft-buffers-per-gate` (by default 5) extra overdraft buffers. -Those buffers are only used, if despite presence of a backpressure, Flink can not stop producing more records to the output. -This can happen in situations like: +For each output subtask can also request up to `taskmanager.network.memory.max-overdraft-buffers-per-gate` +(by default 5) extra overdraft buffers. Those buffers are only used, if the subtask is backpressured +by downstream subtasks but the subtask can not gracefully pause its current process. This can happen +in situations like: - Serializing very large records, that do not fit into a single network buffer. - Flat Map like operator, that produces many output records per single input record. -- Operators that output many records either periodically or on a reaction to some event (for example `WindowOperator`). - -Without overdraft buffers in such situations Flink subtask thread would block on the backpressure, preventing for example unaligned checkpoints -from being triggered. To mitigate this, the overdraft buffers concept has been added. Those buffers are strictly optional and Flink can -make progress even if the Task Manager doesn't have any spare buffers in the global pool to be used as overdraft buffers. +- Operators that output many records either periodically or on a reaction to some events (for +example `WindowOperator`'s triggers). + +Without overdraft buffers in such situations Flink subtask thread would block on the backpressure, +preventing for example unaligned checkpoints from completing. To mitigate this, the overdraft +buffers concept has been added. Those overdraft buffers are strictly optional and Flink can +gradually make progress using only regular buffers, which means `0` is an acceptable configuration +for the `taskmanager.network.memory.max-overdraft-buffers-per-gate`. ## 缓冲区的数量 diff --git a/docs/content/docs/deployment/memory/network_mem_tuning.md b/docs/content/docs/deployment/memory/network_mem_tuning.md index a29cdfa133d..00f8159e016 100644 --- a/docs/content/docs/deployment/memory/network_mem_tuning.md +++ b/docs/content/docs/deployment/memory/network_mem_tuning.md @@ -122,16 +122,20 @@ Unlike the input buffer pool, the configured amount of exclusive buffers and flo #### Overdraft buffers -For each output subtask can also request up to `taskmanager.network.memory.max-overdraft-buffers-per-gate` (by default 5) extra overdraft buffers. -Those buffers are only used, if despite presence of a backpressure, Flink can not stop producing more records to the output. -This can happen in situations like: +For each output subtask can also request up to `taskmanager.network.memory.max-overdraft-buffers-per-gate` +(by default 5) extra overdraft buffers. Those buffers are only used, if the subtask is backpressured +by downstream subtasks but the subtask can not gracefully pause its current process. This can happen +in situations like: - Serializing very large records, that do not fit into a single network buffer. - Flat Map like operator, that produces many output records per single input record. -- Operators that output many records either periodically or on a reaction to some event (for example `WindowOperator`). - -Without overdraft buffers in such situations Flink subtask thread would block on the backpressure, preventing for example unaligned checkpoints -from being triggered. To mitigate this, the overdraft buffers concept has been added. Those buffers are strictly optional and Flink can -make progress even if the Task Manager doesn't have any spare buffers in the global pool to be used as overdraft buffers. +- Operators that output many records either periodically or on a reaction to some events (for +example `WindowOperator`'s triggers). + +Without overdraft buffers in such situations Flink subtask thread would block on the backpressure, +preventing for example unaligned checkpoints from completing. To mitigate this, the overdraft +buffers concept has been added. Those overdraft buffers are strictly optional and Flink can +gradually make progress using only regular buffers, which means `0` is an acceptable configuration +for the `taskmanager.network.memory.max-overdraft-buffers-per-gate`. ## The number of in-flight buffers
