[
https://issues.apache.org/jira/browse/FLINK-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17035153#comment-17035153
]
Xintong Song commented on FLINK-15948:
--------------------------------------
bq. It may be better in client logs since it will be easier to be noticed by
users.
Are you suggesting checking yarn min allocation against Flink's {{
TaskExecutorProcessSpec }} on client side?
bq. Currently, some external job lifecycle management system could not get the
accurate container size.
Could the external system get container size metrics from Yarn/K8s?
> Resource will be wasted when the task manager memory is not a multiple of
> Yarn minimum allocation
> -------------------------------------------------------------------------------------------------
>
> Key: FLINK-15948
> URL: https://issues.apache.org/jira/browse/FLINK-15948
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
> Affects Versions: 1.10.0
> Reporter: Yang Wang
> Priority: Major
>
> If the {{taskmanager.memory.process.size}} is set to 2000m and the Yarn
> minimum allocation is 128m, we will get a container with 2048m. Currently,
> {{TaskExecutorProcessSpec}} is built with 2000m, so we will have 48m wasted
> and they could not be used by Flink.
> I think Flink has accounted all the jvm heap, off-heap, overhead resources.
> So we should not leave these free memory there. And i suggest to update the
> {{TaskExecutorProcessSpec}} according to the Yarn allocated container.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
