[
https://issues.apache.org/jira/browse/FLINK-14936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16983134#comment-16983134
]
Zhu Zhu commented on FLINK-14936:
---------------------------------
I like the idea to introduce the {{LimittingMemoryManager}}! It can prevent
invalid managed memory allocations(operator allocate/reserve more managed
memory than it have declared).
The main concern is that we may not be able to finish the work on it in 1.10,
especially considering the work to apply it to {{DataSet}} operators.
So I think we can have another ticket to track the proposal of
{{LimittingMemoryManager}}.
For this ticket, I think we add a
{{MemoryManagerUtils#computeMemorySize(fraction, memoryType, memoryManager)}}
as well as a {{AbstractStreamOperator#computeMemorySize()}}.
> Introduce MemoryManager#computeMemorySize to calculate managed memory from a
> fraction
> -------------------------------------------------------------------------------------
>
> Key: FLINK-14936
> URL: https://issues.apache.org/jira/browse/FLINK-14936
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 1.10.0
> Reporter: Zhu Zhu
> Priority: Major
>
> A MemoryManager#computeMemorySize(double fraction) is needed to calculate
> managed memory bytes from a fraction.
> It can be helpful for operators to get the memory size it can reserve and for
> further #reserveMemory. (Similar to #computeNumberOfPages).
> Here are two cases that may need this method in near future:
> 1. [Python operator memory
> management|https://lists.apache.org/thread.html/dd4dedeb9354c2ee559cd2f15629c719853915b5efb31a0eafee9361@%3Cdev.flink.apache.org%3E]
> 2. [Statebackend memory
> management|https://issues.apache.org/jira/browse/FLINK-14883]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
