Yang Wang commented on FLINK-15948:

[~xintongsong] Thanks for your comments.

Now i think it makes sense to me to add the exact division check in 
{{YarnClusterDescriptor#validateClusterResources}}. Currently, it is enough for 
our users. If they really want to get the container resource, they could refer 
to the Yarn ResourceManager web or rest api.

> Resource will be wasted when the task manager memory is not a multiple of 
> Yarn minimum allocation
> -------------------------------------------------------------------------------------------------
>                 Key: FLINK-15948
>                 URL: https://issues.apache.org/jira/browse/FLINK-15948
>             Project: Flink
>          Issue Type: Bug
>          Components: Deployment / YARN
>    Affects Versions: 1.10.0
>            Reporter: Yang Wang
>            Priority: Major
> If the {{taskmanager.memory.process.size}} is set to 2000m and the Yarn 
> minimum allocation is 128m, we will get a container with 2048m. Currently, 
> {{TaskExecutorProcessSpec}} is built with 2000m, so we will have 48m wasted 
> and they could not be used by Flink.
> I think Flink has accounted all the jvm heap, off-heap, overhead resources. 
> So we should not leave these free memory there. And i suggest to update the 
> {{TaskExecutorProcessSpec}} according to the Yarn allocated container.

This message was sent by Atlassian Jira

Reply via email to