[
https://issues.apache.org/jira/browse/YARN-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13941725#comment-13941725
]
Rajesh Balamohan commented on YARN-1775:
----------------------------------------
[~vinodkv]
- Sure, it would be great if this can be accommodated in ProcfsBasedProcessTree
itself. I will add a config flag to enable/disable smap based computation
(default is disabled). I will upload the patch soon.
- To get realistic RSS of a process, we need Private_Clean + Private_Dirty +
Shared_Dirty of the process (i.e shared_dirty/n where n is the number of
processes sharing). If we sum up all Shared_Dirty of memory regions, we would
be double counting across processes. If we take PSS, then we would end up
counting Shared_Clean as well. So closer approximation of real shared_dirty
can be obtained by doing Min(shared_dirty, PSS) = Min (shared_dirty,
(shared_dirty+shared_clean)/n); This will fall somewhere between shared_dirty
& shared_dirty/n. Hence, closer approximation of RSS = Math.min(sharedDirty,
pss) + privateDirty + privateClean.
[~cnauroth]
>>
Also interesting would be confirming that containers still get killed for
exceeding the limit with private/non-shared pages.
>>
Had a offline discussion with Gopal. Based on the suggestion, I tried to
ByteBuffer.allocateDirect() which would create enough PRIVATE_DIRTY in the
process. The process got killed once it exceeded the physical memory limits of
the container.
> Create SMAPBasedProcessTree to get PSS information
> --------------------------------------------------
>
> Key: YARN-1775
> URL: https://issues.apache.org/jira/browse/YARN-1775
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: YARN-1775-v2.patch, yarn-1775-2.4.0.patch
>
>
> Create SMAPBasedProcessTree (by extending ProcfsBasedProcessTree), which will
> make use of PSS for computing the memory usage.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
