On 16 December 2011 16:03, Hans Drexler <[email protected]> wrote: > Thanks for your reply. We installed munin on the nodes and noticed that not > all memory was being used by hadoop. So we think we can make it faster by > allocating more memory. But maybe we have been changing the wrong paremeters. > > By the way, technically we did succeed in increasing the HADOOP_HEAPSIZE > setting by changing the whirr-cdh-0.6.0-incubating.jar file inside the jar. > But I still have the feeling we do it the "wrong" way.
Changing HADOOP_HEAPSIZE will not change how much memory the map and reduce task will use (200mb by default). Again do use the suggested parameters to modify the default memory allocation. > PPS. We use this home-grown shell script to deploy munin on all hadoop nodes. > It reads a file cluster-nodes.txt that must contain ip-addres (or full host > name) and password of each node. Munin helps us keeping tabs on what goes on > at the cluster nodes. Maybe somebody can use it. Any remarks welcome (yes, I > know sshpass is dirty!) > > > > #!/bin/sh > while read line; do > set -- $line > host=$1 > pass=$2 > hostname=`echo $host | tr '.' "-"` > echo "Installing munin node on $host" > sshpass -p "$pass" ssh "root@$host" -o StrictHostKeyChecking=no > /usr/bin/aptitude -y install munin-node < /dev/null > sshpass -p "$pass" ssh "root@$host" -o StrictHostKeyChecking=no '/bin/echo > "allow ^50\.57\.191\.88$" >> /etc/munin/munin-node.conf' < /dev/null > sshpass -p "$pass" ssh "root@$host" -o StrictHostKeyChecking=no > '/usr/sbin/service munin-node restart' < /dev/null > echo "Adding $hostname to local config" > rm "/etc/munin/munin-conf.d/conf-$hostname.conf" > echo "[$hostname.localdomain]" > > "/etc/munin/munin-conf.d/conf-$hostname.conf" > echo "\taddress $host" >> "/etc/munin/munin-conf.d/conf-$hostname.conf" > echo "\tuse_node_name yes" >> "/etc/munin/munin-conf.d/conf-$hostname.conf" > echo "Done installing on $host" > done < cluster-nodes.txt > > > > Kind regards, > > Hans >
