Re: Load distribution through the cluster

2017-09-20 Thread Chesnay Schepler
It should only apply to the map operator. On 19.09.2017 17:38, AndreaKinn wrote: If I apply a sharing slot as in the example: DataStream LTzAccStream = env .addSource(new FlinkKafkaConsumer010<>("topic", new CustomDeserializer(), properties))

Re: Load distribution through the cluster

2017-09-19 Thread AndreaKinn
If I apply a sharing slot as in the example: DataStream LTzAccStream = env .addSource(new FlinkKafkaConsumer010<>("topic", new CustomDeserializer(), properties)) .assignTimestampsAndWatermarks(new CustomTimestampExtractor())

Re: Load distribution through the cluster

2017-09-19 Thread Fabian Hueske
There is no notion of "full" in Flink except that one slot will run at most one subtask of each operator. The scheduling depends on the structure of the job, the parallelism of the operators, and the number of slots per TM. It's hard to tell without knowing the details. 2017-09-19 11:57

Re: Load distribution through the cluster

2017-09-19 Thread AndreaKinn
So Flink use the other nodes just if one is completely "full" ? -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Load distribution through the cluster

2017-09-18 Thread AndreaKinn
Hi, I'm experimenting a bit with the cluster. I didn't set any options about sharing slots and chains hoping that Flink decided autonomously how to balance the load through the nodes of the cluster. My cluster is composed by one job and task manager and two task manager. I noted that every time