Hi Alan,
There is a copy of Chukwa Agent running on the cluster. You should try to
shut them down first. You can use bin/stop-agents.sh to shutdown the
previous copy or kill -TERM {pid of chukwa agent} on each of the machines.
Regards,
Eric
On 5/24/10 6:20 AM, "Ratner, Alan S (IS)" <alan.rat...@ngc.com> wrote:
> Eric,
>
> Thanks for the help with the NoClassDefFoundError:
> org/apache/hadoop/metrics/Updater error. I modified the env file and reset
> everything (ran stop-agents, ran stop-collectors, deleted /tmp/chukwa*, then
> copied the chukwa directory to each agent node and deleted /tmp/chukw* on each
> agent node). But now I get a ³agent started already on this machine with
> same portno; bailing out² error from each agent node. Does this mean I¹m still
> doing something wrong?
>
> n...@hadoop1:~/chukwa-0.4.0$ bin/start-agents.sh
> 10.64.147.3: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop2.out
> 10.64.147.8: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop7.out
> 10.64.147.7: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop6.out
> 10.64.147.2: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop1.out
> 10.64.147.4: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop3.out
> 10.64.147.10: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop9.out
> 10.64.147.14: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop13.out
> 10.64.147.18: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop17.out
> 10.64.147.20: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop19.out
> 10.64.147.24: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop23.out
> 10.64.147.27: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop26.out
> 10.64.147.17: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop16.out
> 10.64.147.23: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop22.out
> 10.64.147.41: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop40.out
> 10.64.147.40: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop39.out
> 10.64.147.13: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop12.out
> 10.64.147.31: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop30.out
> 10.64.147.9: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop8.out
> 10.64.147.12: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop11.out
> 10.64.147.11: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop10.out
> 10.64.147.25: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop24.out
> 10.64.147.35: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop34.out
> 10.64.147.22: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop21.out
> 10.64.147.30: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop29.out
> 10.64.147.38: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop37.out
> 10.64.147.29: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop28.out
> 10.64.147.28: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop27.out
> 10.64.147.21: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop20.out
> 10.64.147.37: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop36.out
> 10.64.147.34: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop33.out
> 10.64.147.39: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop38.out
> 10.64.147.15: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop14.out
> 10.64.147.16: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop15.out
> 10.64.147.36: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop35.out
> 10.64.147.32: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop31.out
> 10.64.147.26: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop25.out
> 10.64.147.42: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop41.out
> 10.64.147.19: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop18.out
> 10.64.147.33: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop32.out
> 10.64.147.5: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop4.out
> 10.64.147.3: agent started already on this machine with same portno; bailing
> out
> 10.64.147.8: agent started already on this machine with same portno; bailing
> out
> 10.64.147.7: agent started already on this machine with same portno; bailing
> out
> 10.64.147.2: agent started already on this machine with same portno; bailing
> out
> 10.64.147.4: agent started already on this machine with same portno; bailing
> out
> 10.64.147.10: agent started already on this machine with same portno; bailing
> out
> 10.64.147.14: agent started already on this machine with same portno; bailing
> out
> 10.64.147.18: agent started already on this machine with same portno; bailing
> out
> 10.64.147.20: agent started already on this machine with same portno; bailing
> out
> 10.64.147.24: agent started already on this machine with same portno; bailing
> out
> 10.64.147.15: agent started already on this machine with same portno; bailing
> out
> 10.64.147.17: agent started already on this machine with same portno; bailing
> out
> 10.64.147.27: agent started already on this machine with same portno; bailing
> out
> 10.64.147.39: agent started already on this machine with same portno; bailing
> out
> 10.64.147.41: agent started already on this machine with same portno; bailing
> out
> 10.64.147.40: agent started already on this machine with same portno; bailing
> out
> 10.64.147.23: agent started already on this machine with same portno; bailing
> out
> 10.64.147.31: agent started already on this machine with same portno; bailing
> out
> 10.64.147.11: agent started already on this machine with same portno; bailing
> out
> 10.64.147.13: agent started already on this machine with same portno; bailing
> out
> 10.64.147.9: agent started already on this machine with same portno; bailing
> out
> 10.64.147.12: agent started already on this machine with same portno; bailing
> out
> 10.64.147.25: agent started already on this machine with same portno; bailing
> out
> 10.64.147.22: agent started already on this machine with same portno; bailing
> out
> 10.64.147.35: agent started already on this machine with same portno; bailing
> out
> 10.64.147.30: agent started already on this machine with same portno; bailing
> out
> 10.64.147.38: agent started already on this machine with same portno; bailing
> out
> 10.64.147.29: agent started already on this machine with same portno; bailing
> out
> 10.64.147.28: agent started already on this machine with same portno; bailing
> out
> 10.64.147.21: agent started already on this machine with same portno; bailing
> out
> 10.64.147.34: agent started already on this machine with same portno; bailing
> out
> 10.64.147.37: agent started already on this machine with same portno; bailing
> out
> 10.64.147.16: agent started already on this machine with same portno; bailing
> out
> 10.64.147.36: agent started already on this machine with same portno; bailing
> out
> 10.64.147.32: agent started already on this machine with same portno; bailing
> out
> 10.64.147.26: agent started already on this machine with same portno; bailing
> out
> 10.64.147.19: agent started already on this machine with same portno; bailing
> out
> 10.64.147.42: agent started already on this machine with same portno; bailing
> out
> 10.64.147.33: agent started already on this machine with same portno; bailing
> out
> 10.64.147.5: agent started already on this machine with same portno; bailing
> out
> n...@hadoop1:~/chukwa-0.4.0$
>
> When I check to see what is running on one of the agent nodes I see the agent
> running:
> n...@hadoop6:~$ jps
> 5995 DataNode
> 6147 TaskTracker
> 25642 ChukwaAgent
> 29169 Jps
>
> On the collector node I see:
> n...@hadoop1:~$ jps
> 17295 Jps
> 26355 Main
> 21274 SecondaryNameNode
> 26114 Main
> 7339 ChukwaAgent
> 31259 QuorumPeerMain
> 20978 NameNode
> 25873 Main
> 18000 org.eclipse.equinox.launcher_1.0.201.R35x_v20090715.jar
> 21380 JobTracker
> 31339 ZooKeeperMain
>
>
>
>
> From: Eric Yang [mailto:ey...@yahoo-inc.com]
> Sent: Friday, May 21, 2010 3:20 PM
> To: chukwa-user@hadoop.apache.org
> Subject: Re: NoClassDefFoundError: org/apache/hadoop/metrics/Updater
>
> Hi Alan,
>
> I think you need to be more explicit with the hadoop-0.20.2-core.jar. The
> wild card doesn¹t work. The example config was a little misleading. Sorry
> about that.
>
> Regards,
> Eric
>
> On 5/21/10 7:48 AM, "Ratner, Alan S (IS)" <alan.rat...@ngc.com> wrote:
> I have chukwa 0.4.0 and hadoop 0.20.2 installed on a small cluster. When I
> run bin/start-agents.sh I get the following:
>
> (much more like this above)
>
> 10.64.147.22: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop21.out
>
> 10.64.147.30: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop29.out
>
> 10.64.147.27: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop26.out
>
> 10.64.147.40: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop39.out
>
> 10.64.147.39: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop38.out
>
> 10.64.147.37: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop36.out
>
> 10.64.147.31: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop30.out
>
> 10.64.147.34: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop33.out
>
> 10.64.147.36: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop35.out
>
> 10.64.147.33: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop32.out
>
> 10.64.147.38: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop37.out
>
> 10.64.147.35: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop34.out
>
> 10.64.147.41: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop40.out
>
> 10.64.147.42: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop41.out
>
> 10.64.147.5: starting agent, logging to
> /tmp/chukwa/log/chukwa-chukwa-agent-hadoop4.out
>
> 10.64.147.8: Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/metrics/Updater
>
> 10.64.147.8: at java.lang.ClassLoader.defineClass1(Native Method)
>
> 10.64.147.8: at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
>
> 10.64.147.8: at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
>
> 10.64.147.8: at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
>
> 10.64.147.8: at
> java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
>
> 10.64.147.8: at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
>
> 10.64.147.8: at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
>
> 10.64.147.8: at java.security.AccessController.doPrivileged(Native Method)
>
> 10.64.147.8: at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>
> 10.64.147.2: Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/metrics/Updater
>
> 10.64.147.2: at java.lang.ClassLoader.defineClass1(Native Method)
>
> 10.64.147.2: at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
>
> 10.64.147.2: at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
>
> 10.64.147.2: at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
>
> 10.64.147.2: at
> java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
>
> 10.64.147.2: at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
>
> 10.64.147.2: at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
>
> 10.64.147.2: at java.security.AccessController.doPrivileged(Native Method)
>
> 10.64.147.2: at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>
> (much more like this below)
>
> A similar error reported last year seemed to be related to the Hadoop path.
> This is what I have changed from the chukwa-env.sh defaults:
>
> export HADOOP_HOME="/home/ngc/hadoop-0.20.2"
>
> export HADOOP_CONF_DIR="/home/ngc/hadoop-0.20.2/conf"
>
> export HADOOP_JAR=${HADOOP_HOME}/hadoop-*-core.jar
>
> export JAVA_PLATFORM=Linux-amd64-64
>
> Any idea as to what I am doing wrong?
>
> Alan
>
