Sheetal Dolas created MAPREDUCE-5393:
----------------------------------------
Summary: Physical memory and Virtual Memory counters in Job
Counters display incorrect values
Key: MAPREDUCE-5393
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5393
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 1.2.0
Environment: Linux 5.x
Reporter: Sheetal Dolas
The counters Physical memory (bytes) snapshot - represented by
PHYSICAL_MEMORY_BYTES ) and Virtual memory (bytes) snapshot - represented by
VIRTUAL_MEMORY_BYTES have multiple issues
1. It calculates it as MemTotal from /proc/meminfo which is host level metric
and not task level
2. Even if it is calculated as host level metric, the numbers shown in two
different tasks on same host are different and way different than the MemTotal
displayed in /proc/meminfo if you manually check it. E.g. on a host having 48G
memory the tasks show the value of physical memory (byte) snapshot as
214,142,976 and the number varies every time the job is run.
What was the intention of these counters? Why would we need to get the metric
for every job (as they should be more or less static for given host).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
