Xintong Song created FLINK-17061:
------------------------------------
Summary: Unset process/flink memory size from configuration once
dynamic worker resource is activated.
Key: FLINK-17061
URL: https://issues.apache.org/jira/browse/FLINK-17061
Project: Flink
Issue Type: Task
Components: Runtime / Configuration, Runtime / Coordination
Affects Versions: 1.11.0
Reporter: Xintong Song
With FLINK-14106, memory of a TaskExecutor is decided in two steps on active
resource managers.
- {{SlotManager}} decides {{WorkerResourceSpec}}, including memory used by
Flink tasks: task heap, task off-heap, network and managed memory.
- {{ResourceManager}} derives {{TaskExecutorProcessSpec}} from
{{WorkerResourceSpec}} and the configuration, deciding sizes of memory used by
Flink framework and JVM: framework heap, framework off-heap, jvm metaspace and
jvm overhead.
This works fine for now, because both {{WorkerResourceSpec}} and
{{TaskExecutorProcessSpec}} are derived from the same configurations. However,
it might cause problem if later we have new {{SlotManager}} implementations
that decides {{WorkerResourceSpec}} dynamically. In such cases, the
process/flink sizes in configuration should be ignored, or it may easily lead
to configuration conflicts.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
