[
https://issues.apache.org/jira/browse/FLINK-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17034250#comment-17034250
]
Xintong Song commented on FLINK-15959:
--------------------------------------
[~liuyufei]
For FLIP-56, we might need something like minimum registered TM resources
(memory / cpu cores). Since FLIP-56 is postponed, I think we can introduce this
config option as minimum slots atm. If later FLIP-56 is finished and we decided
to switch to min memory / cpu, we can support backwards compatibility by
automatically converting min slots into the resources.
> Add TaskExecutor number option in FlinkYarnSessionCli
> -----------------------------------------------------
>
> Key: FLINK-15959
> URL: https://issues.apache.org/jira/browse/FLINK-15959
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Coordination
> Affects Versions: 1.11.0
> Reporter: YufeiLiu
> Priority: Major
>
> Flink removed `-n` option after FLIP-6, change to ResourceManager start a new
> worker when required. But I think maintain a TaskExecutor number option is
> necessary. These workers will start immediately when ResourceManager starts
> and would not release even if all slots are free.
> Here are some resons:
> # Users actually know how many resources are needed when run a single job,
> initialize all workers when cluster starts can speed up startup process.
> # Job schedule in topology order, next operator won't schedule until prior
> execution slot allocated. The TaskExecutors will start in several batchs in
> some cases, it might slow down the startup speed.
> # Flink support
> [FLINK-12122|https://issues.apache.org/jira/browse/FLINK-12122] [Spread out
> tasks evenly across all available registered TaskManagers], but it will only
> effect if all TMs are registered. Start all TMs at begining can slove this
> problem.
> *suggestion:*
> I only changed YarnResourceManager, start all container in `initialize`
> stage, and don't comlete slot request until minimum number of slots are
> registered.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
