The values show you the maximum heap size and currently used heap of the job tracker, not running jobs. Furthermore, the HADOOP_HEAPSIZE setting only sets the maximum heap for the daemons, not the tasks in your job.
If you're getting OOMEs, you should add a setting to your mapred-site.xml file that looks like this: <property> <name>mapred.child.java.opts</name> <value>-Xmx1g</value> </property> -Joey On Jun 1, 2011, at 7:35, Ken Williams <[email protected]> wrote: > > > Hi All, > > I'm a bit confused about the values displayed on the 'jobtracker.jsp' page. > In particular, there's a section called 'Cluster Summary'. > > I'm running a small 4-machine Hadoop cluster, and when I point a web-browser > at my master machine (http://master:50030/jobtracker.jsp) it displays, > > Cluster Summary (Heap Size is 15.5 MB / 1.89 GB) > > What exactly do these figures mean ? > > I know that the second figure (1.89 GB) is determined by the value of > the HADOOP_HEAPSIZE variable set in 'conf/hadoop-env.sh'. What I'm not > sure about is exactly what it means, or where the first value (15.5 MB) is > determined > or what it means. > > I'm guessing the 1.89 GB is the amount of heap-memory allocated to Hadoop > on each machine in the cluster. (Correct ?) > > I have no idea what the 15.5 MB means or where it comes from. It never > changes, > not even when a job is running, and I can't find any explanation in the > documentation. > > This page, https://issues.apache.org/jira/browse/HADOOP-4435, seems to > suggest > that the 15.5 MB should be the amount of heap memory currently in use but > since > this value never changes - not even when a job is running and I refresh the > page > - I'm not convinced this is working. > > I'm asking this question because I have a Mahout job which slowly comes to > halt with > a lot of 'OutOfMemoryError: Java heap space' errors, before it is 'Killed'. > > I'm using Hadoop 0.20.2 and the latest Mahout snapshot version. > > Thanks for any help. > > Ken > >
