Re: Question about scheduling in 0.7

2015-02-23 Thread Stephan Ewen
In addition, to what Fabian wrote: Yes, one slot can run multiple Tasks. In the batch API, one slot can run concurrently one task of each operation (for example one source, one mapper, one reducer, one sink). On Mon, Feb 23, 2015 at 10:38 PM, Fabian Hueske wrote: > Hi Nico, > > yes, Flink runs

Re: Question about scheduling in 0.7

2015-02-23 Thread Fabian Hueske
Hi Nico, yes, Flink runs tasks in threads. Each TaskManager runs in its own JVM, everything within a TaskManager is parallelized in threads. Since a TM can offer multiple slots, also tasks across slots run in the same JVM and in different threads. Flink has a pipelined processing model, which mea

Question about scheduling in 0.7

2015-02-23 Thread Nico Scherer
Hi all, we’re currently playing/messing around with scheduling in flink 0.7. We found out that if we run a single job with a certain degree of parallelism, multiple tasks/vertices are executed within a single task manager at the same time (or at least before the prior stage is switched to finis