Understood, thank you for the quick response! Thanks, Li
Xintong Song <tonysong...@gmail.com> 于2019年12月25日周三 下午5:05写道: > Hi Li, > > It is true that currently all the task managers have the same heap size, > and it's fixed ever since started. Unfortunately, your needs cannot be > satisfied at the moment. > > Heap size of task managers cannot be changed once started, because flink > task managers run in JVMs and JVM does not support resizing after started. > > However, there is an ongoing approach towards your needs. The community is > working on fine-grained resource management, which in general allows > specify per task/operator resource requirements and allocate task manager > resources accordingly. You can refers to FLIP-53 [1] and FLIP-56 [2] for > more details. Another related effort is pluggable slot manager [3], which > allows having pluggable resource scheduling strategies such as launch task > managers with customized resources according to the tasks' requirements. > > Thank you~ > > Xintong Song > > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-53%3A+Fine+Grained+Operator+Resource+Management > [2] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-56%3A+Dynamic+Slot+Allocation > [3] https://issues.apache.org/jira/browse/FLINK-14106 > > On Wed, Dec 25, 2019 at 4:18 PM Li Zhao <ilo...@gmail.com> wrote: > >> Hi, >> >> Greetings, hope this is the proper place to ask questions, apologize if >> not. >> >> We have a shared flink cluster running with docker, want to set different >> heap size per task(some tasks require larger heap size, while most tasks >> only need a little), is it feasible? >> >> I've gone through [1], [2] and [3], my current understanding is all task >> managers have the same heap size which is set by `taskmanager.heap.size` >> and is fixed ever since the task manager is started, and all tasks running >> in that task manager will share that heap size. >> >> Am I understanding it correctly? And any approaches to our needs? >> >> Thanks in advance! >> >> [1]: >> https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/cli.html >> [2]: >> https://ci.apache.org/projects/flink/flink-docs-release-1.9/concepts/runtime.html >> [3]: >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors >> >> Thanks, >> Li >> >