[
https://issues.apache.org/jira/browse/FLINK-14566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963824#comment-16963824
]
Zhu Zhu commented on FLINK-14566:
---------------------------------
I think we can do it if there is assumption that an operator with UNKOWN
resource should always work with little managed memory.
Or if we can, in the first version, accept the limitation that an operator with
UNKOWN resource might break in the case of little managed memory available,
which may happen if there are too many operators in one slot sharing
group(logical pipelined region).
[~ykt836] what do you think?
> Enable ResourceSpec to get/set whether managed memory is used
> -------------------------------------------------------------
>
> Key: FLINK-14566
> URL: https://issues.apache.org/jira/browse/FLINK-14566
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Affects Versions: 1.10.0
> Reporter: Zhu Zhu
> Priority: Major
> Fix For: 1.10.0
>
>
> To calculate managed memory fraction for an operator with UNKNOWN resources,
> we need to know whether the operator will use managed memory, according to
> FLINK-14062.
> I'd propose to define special values of
> onHeapManagedMemory/offHeapManagedMemory
> in ResourceSpec(introduced in FLINK-14405) to distinguish whether managed
> memory is used.
> - For UNKNOWN resource spec, onHeapManagedMemory==MemorySize.ZERO means on
> heap managed memory is used, while onHeapManagedMemory==null means not. For
> non-UNKNOWN resource spec, managed memory size larger than zero indicates it
> is used.
> - similar for off heap managed memory.
> getters: isUsingManagedMemoryOnHeap()/isUsingManagedMemoryOffHeap()
> setters:
> setUsingManagedMemoryOnHeap(boolean)/setUsingManagedMemoryOffHeap(boolean)
> (setters are only allowed to invoke on UNKNOWN, should be ignored for
> non-UNKNOWN for compatibility)
> {{ResourceSpec#isKnown()}} should be added to replace existing UNKNOWN usages
> for equality checking.
> cc [~ykt836] [~trohrmann] [~azagrebin] [~xintongsong]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
