[
https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439274#comment-15439274
]
Jason Lowe commented on YARN-5551:
----------------------------------
bq. Memory pressure could cause either of them to spill to disk, so it doesn't
make sense to me why we would count them differently.
Hmm. I guess they will behave differently in practice because a file-backed
region of memory could pause the process if it dirties too many pages and
triggers an I/O flush, whereas that's not going to happen if the memory is
simply backed by the system's swap device.
So I suppose ignoring the private dirty field and only focusing on anonymous
pages even with plenty of swap makes sense because we're equating private dirty
with the buffer cache and we don't charge other processes for their use of the
buffer cache.
> Ignore deleted file mapping from memory computation when smaps is enabled
> -------------------------------------------------------------------------
>
> Key: YARN-5551
> URL: https://issues.apache.org/jira/browse/YARN-5551
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Minor
> Attachments: YARN-5551.branch-2.001.patch
>
>
> Currently deleted file mappings are also included in the memory computation
> when SMAP is enabled. For e.g
> {noformat}
> 7f612004a000-7f612004c000 rw-s 00000000 00:10 4201507513
> /dev/shm/HadoopShortCircuitShm_DFSClient_NONMAPREDUCE_-521969216_162_734673185
> (deleted)
> Size: 8 kB
> Rss: 4 kB
> Pss: 2 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 4 kB
> Private_Clean: 0 kB
> Private_Dirty: 0 kB
> Referenced: 4 kB
> Anonymous: 0 kB
> AnonHugePages: 0 kB
> Swap: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> 7fbf28000000-7fbf68000000 rw-s 00000000 08:02 11927571
> /tmp/7298569189125604642/arena-1291157252088664681.cache (deleted)
> Size: 1048576 kB
> Rss: 17288 kB
> Pss: 17288 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 232 kB
> Private_Dirty: 17056 kB
> Referenced: 17288 kB
> Anonymous: 0 kB
> AnonHugePages: 0 kB
> Swap: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> {noformat}
> It would be good to exclude these from getSmapBasedRssMemorySize()
> computation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
