Re: Parallelism questions

I'm not aware of someone working on this feature right now. On Tue, Jan 15, 2019 at 3:22 PM Alexandru Gutan wrote: > Thats great news! > > Are there any plans to expose it in the upcoming Flink release? > > On Tue, 15 Jan 2019 at 12:59, Till Rohrmann wrote: > >> Hi Alexandru, >> >> at the

Re: Parallelism questions

Thats great news! Are there any plans to expose it in the upcoming Flink release? On Tue, 15 Jan 2019 at 12:59, Till Rohrmann wrote: > Hi Alexandru, > > at the moment `/jobs/:jobid/rescaling` will always change the parallelism > for all operators. The maximum is the maximum parallelism which

Re: Parallelism questions

Hi Alexandru, at the moment `/jobs/:jobid/rescaling` will always change the parallelism for all operators. The maximum is the maximum parallelism which you have defined for an operator. I agree that it should also be possible to rescale an individual operator. There internal functionality is

Re: Parallelism questions

Thanks Till! To execute the above (using Kubernetes), one would enter the running JobManager service and execute it? The following REST API call does the same */jobs/:jobid/rescaling*? I assume it changes the base parallelism, but what it will do if I had already set the parallelism of my

Re: Parallelism questions

Hi Alexandru, you can use the `modify` command `bin/flink modify --parallelism ` to modify the parallelism of a job. At the moment, it is implemented as first taking a savepoint, stopping the job and then redeploying the job with the changed parallelism and resuming from the savepoint. Cheers,

Re: Parallelism questions

Hi Alexandru As for 2, generally speaking the number of required slots depends on number of slot sharing groups. By default all operators belong to the default slot sharing group, that means a job requires as many slots as maximal parallelism in the job. More on the distributed runtime you can

Parallelism questions

Hi everyone! 1. Is there a way to increase the parallelism (e.g. through REST) of some operators in a job without re-deploying the job? I found this answer which mentions scaling at runtime on