Re: Use jvm to run flink on single-node machine with many cores

2016-02-23 Thread Ufuk Celebi
On Tue, Feb 23, 2016 at 10:17 AM, Ana M. Martinez wrote: > I believe that setting taskmanager.numberOfTaskSlots is not necessary, but > setParallelism is, as by default 1 was taken. Yes, the number of slots in local execution defaults to the maximum parallelism of the job.

Re: Use jvm to run flink on single-node machine with many cores

2016-02-23 Thread Ana M. Martinez
Hi all, Thank you very much for your help. It worked perfectly like this: Configuration conf = new Configuration(); conf.setInteger("taskmanager.network.numberOfBuffers", 16000); conf.setInteger("taskmanager.numberOfTaskSlots”,32); final ExecutionEnvironment env = ExecutionEnvironment.createLoc

Re: Use jvm to run flink on single-node machine with many cores

2016-02-22 Thread Ufuk Celebi
Note that the method to call in the example should be `conf.setInteger` and the second argument not a String but an int. On Sun, Feb 21, 2016 at 1:41 PM, Márton Balassi wrote: > Dear Ana, > > If you are using a single machine with multiple cores, but need convenient > access to the configuration

Re: Use jvm to run flink on single-node machine with many cores

2016-02-21 Thread Márton Balassi
Dear Ana, If you are using a single machine with multiple cores, but need convenient access to the configuration I would personally recommend using the local cluster option in the flink distribution. [1] If you want to avoid having a flink distro on the machine, then Robert's solution is the way t

Re: Use jvm to run flink on single-node machine with many cores

2016-02-21 Thread Robert Metzger
Hi Ana, you can create a StreamExecutionEnvironment also by passing a configuration object. In the configuration, you can also configure the number of network buffers. // set up the execution environmentConfiguration conf = new Configuration(); conf.setBoolean("taskmanager.network.numberOfBuffer

Use jvm to run flink on single-node machine with many cores

2016-02-21 Thread Ana M. Martinez
Hi all, I am trying to run a program using the flink java library with ExecutionEnvironment.getExecutionEnvironment() from the command line using java -jar. If I run the code in my machine (with four cores) or in a multi-node cluster (using yarn) the program runs normally, but if I want to run