Dear Ana, If you are using a single machine with multiple cores, but need convenient access to the configuration I would personally recommend using the local cluster option in the flink distribution. [1] If you want to avoid having a flink distro on the machine, then Robert's solution is the way to go.
[1] https://ci.apache.org/projects/flink/flink-docs-master/setup/local_setup.html On Sun, Feb 21, 2016 at 1:34 PM, Ana M. Martinez <a...@cs.aau.dk> wrote: > Hi all, > > I am trying to run a program using the flink java library with > ExecutionEnvironment.getExecutionEnvironment() from the command line using > java -jar. > > If I run the code in my machine (with four cores) or in a multi-node > cluster (using yarn) the program runs normally, but if I want to run it on > a machine with a single node and 32 cores using java -jar I get the > following error: > > 02/21/2016 13:33:09 MapPartition (MapPartition at > toBatches(ConversionToBatches.java:55))(29/32) switched to FAILED > java.io.IOException: Insufficient number of network buffers: required 1, > but only 0 available. The total number of network buffers is currently set > to 2048. You can increase this number by setting the configuration key > 'taskmanager.network.numberOfBuffers'. > at > org.apache.flink.runtime.io.network.buffer.NetworkBufferPool.createBufferPool(NetworkBufferPool.java:196) > at > org.apache.flink.runtime.io.network.NetworkEnvironment.registerTask(NetworkEnvironment.java:325) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:488) > at java.lang.Thread.run(Thread.java:745) > > In this case (java -jar), I don’t know if or how I can increase the number > of network buffers. Is there a way to do it without having to use yarn (as > I don’t have hadoop installed)? > > Thanks, > Ana > >
