[
https://issues.apache.org/jira/browse/FLINK-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033407#comment-17033407
]
Xintong Song edited comment on FLINK-15948 at 2/10/20 7:05 AM:
---
Hi [~fly_in_gis],
I'm not sure about overwriting {{ TaskExecutorProcessSpec }} according to Yarn
allocated container, for the following reasons.
- Overwriting {{ TaskExecutorProcessSpec }} means we are deciding task executor
total and individual components' memory sizes not only based on user's
configurations, but also based on the size of container allocated from Yarn.
This is implicit and make the memory configuration hard to understand for users.
- The extra memory (48m in your example) is not necessarily wasted. Although it
is not accounted for in jvm heap size and max direct memory limit, it can still
be used as native memory.
- AFAIK, it is a Yarn specific issue that allocated container may have more
memory than requested. I would try to avoid complicating the common resource
and slot management logics with the assumption that resources of requested /
actual task executors could be different.
I admit that for the major memory components that user cares (jvm heap, managed
memory, network memory), the extra allocated memory cannot be used. Instead of
being too smart and overwrite users' memory configurations, I would suggest to
add an info or warning level log in such cases, to inform users about the
potential memory waste. WDYT?
was (Author: xintongsong):
Hi [~fly_in_gis],
I'm not sure about overwriting {{ TaskExecutorProcessSpec }} according to Yarn
allocated container, for the following reasons.
- Overwriting {{ TaskExecutorProcessSpec }} means we are deciding task executor
total and individual components' memory sizes not only based on user's
configurations, but also based on the size of container allocated from Yarn.
This is implicit and make the memory configuration hard to understand for users.
- The extra memory (48m in your example) is not necessary wasted. Although it
is not accounted for in jvm heap size and max direct memory limit, it can still
be used as native memory.
- AFAIK, it is a Yarn specific issue that allocated container may have more
memory than requested. I would try to avoid complicating the common resource
and slot management logics with the assumption that resources of requested /
actual task executors could be different.
I admit that for the major memory components that user cares (jvm heap, managed
memory, network memory), the extra allocated memory cannot be used. Instead of
being too smart and overwrite users' memory configurations, I would suggest to
add an info or warning level log in such cases, to inform users about the
potential memory waste. WDYT?
> Resource will be wasted when the task manager memory is not a multiple of
> Yarn minimum allocation
> -
>
> Key: FLINK-15948
> URL: https://issues.apache.org/jira/browse/FLINK-15948
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
>Affects Versions: 1.10.0
>Reporter: Yang Wang
>Priority: Major
>
> If the {{taskmanager.memory.process.size}} is set to 2000m and the Yarn
> minimum allocation is 128m, we will get a container with 2048m. Currently,
> {{TaskExecutorProcessSpec}} is built with 2000m, so we will have 48m wasted
> and they could not be used by Flink.
> I think Flink has accounted all the jvm heap, off-heap, overhead resources.
> So we should not leave these free memory there. And i suggest to update the
> {{TaskExecutorProcessSpec}} according to the Yarn allocated container.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
