Hi, I think you've chosen a good initial value for the parallelism. The higher the parallelism, the more network buffers are needed. I would follow the recommendation from the exception and increase the number of network buffers.
On Thu, May 5, 2016 at 11:23 AM, Punit Naik <naik.puni...@gmail.com> wrote: > Hello > > I was running a program with 'parallelism.default' of 384 as I read in the > documentation on Flink's official page that 'parallelism.default' is "the > total number of CPUs in the cluster". I have four machines with 96 cores on > each of them. So 96*4=384. But the program thew an error saying: > > Caused by: java.io.IOException: Insufficient number of network buffers: > required 384, but only 298 available. The total number of network buffers > is currently set to 2048. You can increase this number by setting the > configuration key 'taskmanager.network.numberOfBuffers'. > > What does this mean? And how to choose a proper value for parallelism? > > -- > Thank You > > Regards > > Punit Naik >
