Re: Re: Independence of task parallelism

2021-03-05 Thread Piotr Nowojski
> Jan > > > *Gesendet:* Mittwoch, 03. März 2021 um 19:53 Uhr > *Von:* "Piotr Nowojski" > *An:* "Jan Nitschke" > *Cc:* "user" > *Betreff:* Re: Independence of task parallelism > Hi Jan, > > As far as I remember, Flink does

Aw: Re: Independence of task parallelism

2021-03-05 Thread Jan Nitschke
n Nitschke" Cc: "user" Betreff: Re: Independence of task parallelism Hi Jan,   As far as I remember, Flink doesn't handle very well cases like (1-2-1-1-1) and two Task Managers. There are no guarantees how the operators/subtasks are going to be scheduled, but most likely it wil

Re: Independence of task parallelism

2021-03-03 Thread Piotr Nowojski
Hi Jan, As far as I remember, Flink doesn't handle very well cases like (1-2-1-1-1) and two Task Managers. There are no guarantees how the operators/subtasks are going to be scheduled, but most likely it will be as you mentioned/observed. First task manager will be handling all of the operators,

Independence of task parallelism

2021-02-28 Thread Jan Nitschke
Hello, We are working on a project where we want to gather information about the job performance across different task level parallelism settings. Essentially, we want to see how the throughput of a single task varies across different parallelism settings, e.g. for a job of 5 tasks: 1-1-1-1-1