[
https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437379#comment-15437379
]
Gopal V commented on YARN-5551:
-------------------------------
bq. I guess where I'm getting hung up is on the deleted part. Unless I'm
mistaken, the OS isn't going to care whether the file is deleted or not when
the process still has a mapping to it.
Actually, that's just a safety rail to cut down IO here - when the process
exits, the deleted file pages just disappear.
bq. So in that sense I don't see why we're special-casing deleted files.
We can apply this patch to all file mappings actually - the special-casing was
primarily to cut down the impact of the patch and reduce unintended
consequences.
For non-deleted files, I'd like IO isolation as well (i.e the IO impact lasts
past process-death), but that's a harder problem to solve into the 2.7.x branch
(definitely to be tackled in 3.x and specifically for a modern cgroups setup).
> Ignore deleted file mapping from memory computation when smaps is enabled
> -------------------------------------------------------------------------
>
> Key: YARN-5551
> URL: https://issues.apache.org/jira/browse/YARN-5551
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Minor
> Attachments: YARN-5551.branch-2.001.patch
>
>
> Currently deleted file mappings are also included in the memory computation
> when SMAP is enabled. For e.g
> {noformat}
> 7f612004a000-7f612004c000 rw-s 00000000 00:10 4201507513
> /dev/shm/HadoopShortCircuitShm_DFSClient_NONMAPREDUCE_-521969216_162_734673185
> (deleted)
> Size: 8 kB
> Rss: 4 kB
> Pss: 2 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 4 kB
> Private_Clean: 0 kB
> Private_Dirty: 0 kB
> Referenced: 4 kB
> Anonymous: 0 kB
> AnonHugePages: 0 kB
> Swap: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> 7f6123f99000-7f6163f99000 rw-p 00000000 08:41 211419477
> /grid/4/hadoop/yarn/local/usercache/root/appcache/application_1466700718395_1249/container_e19_1466700718395_1249_01_000003/7389389356021597290.cache
> (deleted)
> Size: 1048576 kB
> Rss: 637292 kB
> Pss: 637292 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 637292 kB
> Referenced: 637292 kB
> Anonymous: 637292 kB
> AnonHugePages: 0 kB
> Swap: 0 kB
> KernelPageSize: 4 kB
> {noformat}
> It would be good to exclude these from getSmapBasedRssMemorySize()
> computation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
