[
https://issues.apache.org/jira/browse/FLINK-15037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16988444#comment-16988444
]
Zhu Zhu commented on FLINK-15037:
---------------------------------
Thanks for the feedback [~ykt836]. This task is not for 1.10. We will review
the design later regarding a few other requirements:
1. shared managed memory for state backends. With FLINK-7289, RocksDb instances
in the same slot will share the same LRUCache. LRUCache would be the only
managed memory consumer (in streaming job cases) and it will take all the
managed memory from a slot. So that it requires slot wide
2. floating managed memory for operators as you mentioned
> Introduce LimittingMemoryManager as operator scope MemoyManager
> ---------------------------------------------------------------
>
> Key: FLINK-15037
> URL: https://issues.apache.org/jira/browse/FLINK-15037
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 1.10.0
> Reporter: Zhu Zhu
> Priority: Major
>
> Current {{MemoryManager}} is slot scope component, and the operators needs to
> use a fraction to compute memory size/pages it can allocate and use it to
> reserver memory or allocate pages.
> This, however, requires operators to be aware of the managed memory fraction.
> There is also risk that one operator may over allocate resources than it has
> declared and causes other operators to break.
> To separate concerns, we can introduce a {{LimittingMemoryManager}} which
> wraps the the original MemoryManager but limit the available memory size to
> the fraction of total memory governed by the original one. This wrapper would
> be a operator scope {{MemoryManager}}.
> Some more detailed discussions can be found in FLINK-14936.
> cc [~azagrebin]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
