Hi, Pritam.
This error message indicates that the current configuration of the
network buffer is not enough to handle the current workload.
> What is the meaning of this exception (The total number of network buffers
> is currently set to 22773 of 32768 bytes each)?
>
This just provides some information about the current status of network
buffers (22773 * 32768 bytes ~= 711MB).
How to figure out a good combination of
> ('taskmanager.memory.network.fraction', 'taskmanager.memory.network.min',
> and 'taskmanager.memory.network.max'.)
> for this issue ?
>
IIUC, There is no absolute standard for setting these parameters.
These parameters may be affected by many factors, such as the data flow
rate, computational complexity, and memory usage of your job.
Some steps to setup and adjust these parameters:
1. Check the available memory on your job.
2. Evaluate the network usage and consider how much memory could be used
for network buffers.
3. Monitor the system and collect metrics such as network throughput,
memory usage.
4. Adjust these parameters if the job has a high network usage or
memory-intensive.
Just a personal and immature suggestion about how to adjust when it's not
enough: 1. Increase taskmanager.memory.network.fraction from 0.1 to 0.2,
or just increase taskmanager.memory.network.max slightly.
2. If the buffer size is too large, it may affect checkpoints. So it's
recommended to combine with buffer debloating.
<https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/memory/network_mem_tuning/#the-buffer-debloating-mechanism>
On Tue, Jun 6, 2023 at 2:44 AM Pritam Agarwala <pritamagarwala...@gmail.com>
wrote:
> Hi All,
>
>
> java.io.IOException: Insufficient number of network buffers: required 2,
> but only 0 available. The total number of network buffers is currently set
> to 22773 of 32768 bytes each.
>
> What is the meaning of this exception (The total number of network buffers
> is currently set to 22773 of 32768 bytes each)?
>
> How to figure out a good combination of
> ('taskmanager.memory.network.fraction', 'taskmanager.memory.network.min',
> and 'taskmanager.memory.network.max'.)
> for this issue ?
>
>
> Thanks & Regards,
> Pritam
>
--
Best,
Hangxiang.
