[
https://issues.apache.org/jira/browse/MESOS-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14218322#comment-14218322
]
Ian Downes commented on MESOS-2104:
-----------------------------------
My understanding is that Eric was taking a very specific interpretation of rss
as being all the pages a cgroup has in memory, including pages "like I/O are
also counted". Under that interpretation the resident set includes page cache
and it would correspond to usage_in_bytes, as he states and as is currently
implemented.
However, I don't think that interpretation is intuitive to users. Because
available memory is used for page caching it is expected that usage_in_bytes
will climb up to limit_in_bytes and hover there, with the kernel reclaiming
page_cache pages as necessary if the application requests memory. Therefore the
memory used by the job becomes approximately equivalent to the limit.
What is more informative is the size of the anonymous cache as that really
dictates how close you are to running out of memory and being OOMed (because
there are no pages to reclaim). I posit that users think of "RSS" when they
think of memory usage (hence the naming) and this corresponds to anonymous
cache, i.e., total_rss.
For example, suppose my limit is 10 GB and after my processes have been busy
reading/writing, etc., then I'd expect usage_in_bytes to be near 10 GB,
total_rss might be 8 GB and total_cache 2 GB. If I have a memory leak then that
8 GB will creep up and the 2 GB will creep down. I'd be monitoring the
total_rss and alerting on it reaching a threshold as a fraction of the limit.
The other comments Eric made in regard to the posix cpu isolator are completely
valid - it reports numbers that are accounted in a totally different way.
However, the posix cpu isolator is really only "best effort" and IMHO its
discrepancies should not influence how we report or name the more accurate
numbers available using the memory cgroup controller.
> Correct naming of cgroup memory statistics
> ------------------------------------------
>
> Key: MESOS-2104
> URL: https://issues.apache.org/jira/browse/MESOS-2104
> Project: Mesos
> Issue Type: Improvement
> Components: isolation
> Affects Versions: 0.20.0
> Reporter: Ian Downes
> Assignee: Ian Downes
> Labels: twitter
> Fix For: 0.22.0
>
>
> mem_rss_bytes is *not* RSS but is the total memory usage
> (memory.usage_in_bytes) of the cgroup, including file cache etc. Actual RSS
> is reported as mem_anon_bytes. These, and others, should be consistently
> named.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
