slfan1989 commented on code in PR #8726:
URL: https://github.com/apache/hadoop/pull/8726#discussion_r4093075831
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsV2ResourceCalculator.java:
##########
@@ -110,11 +110,24 @@ public CGroupsV2ResourceCalculator(String pid) {
super(
pid,
Collections.singletonList(CPU_STAT),
- MEM_STAT,
- MEMSW_STAT
+ MEM_STAT
);
}
+ @Override
+ protected long calculateVirtualMemory() {
+ // Unlike cgroup v1, cgroup v2 does not expose a combined memory+swap
+ // counter. The virtual memory has to be derived by adding the swap usage
+ // (memory.swap.current) to the resident anonymous memory
+ // (memory.stat#anon).
+ long anon = getStat(MEM_STAT);
Review Comment:
Could we use `memory.current` for the resident-memory component here rather
than `memory.stat#anon` ?
`anon` only covers anonymous memory, whereas the v1 implementation reads
`memory.memsw.usage_in_bytes`, which includes other charged memory such as page
cache. Using `anon + swap` therefore still undercounts workloads with
substantial non-anonymous memory and does not preserve the v1 accounting
semantics.
For the combined cgroup memory-plus-swap metric, please use `memory.current
+ memory.swap.current` and add a test where `memory.current` is greater than
`anon`, so the difference is covered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
