Caught it in action. Running ps -e -o 'vsz pid ruser args' |sort -nr|head -5 on a machine where the map task was running 04812 16962 sguha /home/godhuli/custom/jdk1.6.0_11/jre/bin/java -Djava.library.path=/home/godhuli/custom/hadoop/bin/../lib/native/Linux-amd64-64:/home/godhuli/custom/hdfs/mapred/local/taskTracker/jobcache/job_200812282102_0003/attempt_200812282102_0003_m_000000_0/work -Xmx200m -Djava.io.tmpdir=/home/godhuli/custom/hdfs/mapred/local/taskTracker/jobcache/job_200812282102_0003/attempt_200812282102_0003_m_000000_0/work/tmp -classpath /attempt_200812282102_0003_m_000000_0/work -Dhadoop.log.dir=/home/godhuli/custom/hadoop/bin/../logs -Dhadoop.root.logger=INFO,TLA -Dhadoop.tasklog.taskid=attempt_200812282102_0003_m_000000_0 -Dhadoop.tasklog.totalLogFileSize=0 org.apache.hadoop.mapred.Child 127.0.0.1 40443 attempt_200812282102_0003_m_000000_0 1525207782
Also, the reducer only used 540mb. I notice -Xmx200m was passed, how to change it? Regards Saptarshi On Sun, Dec 28, 2008 at 10:19 PM, Saptarshi Guha <saptarshi.g...@gmail.com> wrote: > On Sun, Dec 28, 2008 at 4:33 PM, Brian Bockelman <bbock...@cse.unl.edu> wrote: >> Hey Saptarshi, >> >> Watch the running child process while using "ps", "top", or Ganglia >> monitoring. Does the map task actually use 16GB of memory, or is the memory >> not getting set properly? >> >> Brian > > I haven't figured out how to run ganglia, however, also the children > quit before i can see their memory usage. The trackers all use > 16GB.(from the ps command). However, i noticed some use 512MB > only(when i manged to catch them in time) > > Regards > -- Saptarshi Guha - saptarshi.g...@gmail.com
