OK, I see. Thank you very much Song
Best Regards
Chase
On Mon, May 27, 2019 at 4:01 AM Xintong Song wrote:
> Well, it depends on how many resource are needed for one pipeline of you
> job and how many resource are are configured for each TaskExecutor. In
> addition, the resource of each
Well, it depends on how many resource are needed for one pipeline of you
job and how many resource are are configured for each TaskExecutor. In
addition, the resource of each TaskManager also depends on the job's
resource needs and your environment. So having one slot for each
TaskManager would be
Hi Song,
You said "In that way, the total slots (or number of TaskManagers if you
config on slot for each TaskManager)", do you imply that one taskmanager
contains one slot?
Do you have some experience on how many slots to spawn for one TaskManager?
I read the Flip-6, it says "For the sake of
Yes true. I am trying to figure out how the TaskManagers are distributed
across physical machines by Mesos and YARN. Maybe I shoud start a new
thread for help.
Thank you Song
Best,
Pengcheng
On Fri, May 24, 2019 at 10:52 AM Xintong Song wrote:
> As far as I know, Flink does not have any
As far as I know, Flink does not have any requirements on how the
TaskManagers are distributed across physical machines. So I think it really
depends on the scheduling policy of the Mesos cluster. I'm not an expert on
Mesos, so correct me if I was wrong.
Thank you~
Xintong Song
On Fri, May
Hi Song,
Thank you for the clarification.
Now I know TaskManagers are automatically allocated. Yet, I am still not
very clear how the TMs are allocated.
I'm guessing the allocation process would be:
On the job side, I have a job with each operator parallelism=5. Since one
TaskManager has one slot.
Hi black,
If you are running Flink on Yarn or Mesos, Flink will automatically
allocate resource and launch new TaskManagers as needed.
If you are using Flink standalone mode, then the easiest way is to enable
slot sharing and set all the vertices into the same group (which is by
default). In
Hi,
I am redesigning the scheduler of the JobManager to place tasks of a job
across TaskManagers accroding to a scheduling policy.
I am reading the Flip-6 proposal and found that the common case is "one
TaskManager launchs one slot", and "one Flink cluster serves one job". But
I did not find how
