[
https://issues.apache.org/jira/browse/FLINK-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17035102#comment-17035102
]
Yang Wang commented on FLINK-15948:
-----------------------------------
[~xintongsong] Thanks for your comments.
It makes sense to me to add some {{WARN}} logs to inform the users some extra
memory could not be used by Flink because of the taskmanager resource
configuration. I am just thinking where to put the logs. It may be better in
client logs since it will be easier to be noticed by users.
Another concern is maybe we need to provide a metric to get the
jobmanager/taskmanager container size. Currently, some external job lifecycle
management system could not get the accurate container size.
> Resource will be wasted when the task manager memory is not a multiple of
> Yarn minimum allocation
> -------------------------------------------------------------------------------------------------
>
> Key: FLINK-15948
> URL: https://issues.apache.org/jira/browse/FLINK-15948
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
> Affects Versions: 1.10.0
> Reporter: Yang Wang
> Priority: Major
>
> If the {{taskmanager.memory.process.size}} is set to 2000m and the Yarn
> minimum allocation is 128m, we will get a container with 2048m. Currently,
> {{TaskExecutorProcessSpec}} is built with 2000m, so we will have 48m wasted
> and they could not be used by Flink.
> I think Flink has accounted all the jvm heap, off-heap, overhead resources.
> So we should not leave these free memory there. And i suggest to update the
> {{TaskExecutorProcessSpec}} according to the Yarn allocated container.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
