Hi, I think the error might be because I have to start the BSP Master server before trying to run the BSP Master node.
But, when I run the command bin/start-bspd.sh, hama tries to use openjdk for some reason. I have set the default version to sun JDK and my JAVA_HOME points to java-6-sun. Any idea why I might be getting this error ? ashish@ubuntu:~/Desktop/Project/hama-trunk$ bin/start-bspd.sh localhost: starting zookeeper, logging to /home/ashish/Desktop/Project/hama-trunk/bin/../logs/hama-ashish-zookeeper-ubuntu.out localhost: /home/ashish/Desktop/Project/hama-trunk/bin/../bin/hama: line 186: /home/ashish/Desktop/Project/hama-trunk/”/usr/lib/jvm/java-6-openjdk”/bin/java: No such file or directory starting bspmaster, logging to /home/ashish/Desktop/Project/hama-trunk/bin/../logs/hama-ashish-bspmaster-ubuntu.out localhost: starting groom, logging to /home/ashish/Desktop/Project/hama-trunk/bin/../logs/hama-ashish-groom-ubuntu.out localhost: /home/ashish/Desktop/Project/hama-trunk/bin/../bin/hama: line 186: /home/ashish/Desktop/Project/hama-trunk/”/usr/lib/jvm/java-6-openjdk”/bin/java: No such file or directory ashish@ubuntu:~/Desktop/Project/hama-trunk$ Thanks Ashish On Thu, Jun 23, 2011 at 2:49 AM, Edward J. Yoon <[email protected]> wrote: > Hey, > > Let's see the error message. > >> java.net.UnknownHostException: Invalid hostname for server: local > > and > >> at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:248) > > line 248: host = getAddress(conf).getHostName(); > > I think, you have to try to fix/debug your problem yourself before calling. > > On Thu, Jun 23, 2011 at 6:33 PM, Ashish Agarwal <[email protected]> > wrote: >> It didn't work. >> I have attached my hama-site.xml.template file. >> >> >> On Thu, Jun 23, 2011 at 2:16 AM, Edward J. Yoon <[email protected]> >> wrote: >>> Have you tried to run with <value>ubuntu</value>? >>> >>> On Thu, Jun 23, 2011 at 6:04 PM, Ashish Agarwal <[email protected]> >>> wrote: >>>> <property> >>>> <name>fs.default.name</name> >>>> <value>localhost:9000</value> >>>> </property> >>>> >>>> This is the same in both hadoop-site.xml and the hama-site.xml >>>> >>>> The /etc/hosts file looks like this - >>>> >>>> 69.90.54.2 ubuntu # Added by NetworkManager >>>> 127.0.0.1 localhost.localdomain localhost >>>> ::1 ubuntu localhost6.localdomain6 localhost6 >>>> 127.0.1.1 ubuntu.ubuntu-domain ubuntu >>>> >>>> # The following lines are desirable for IPv6 capable hosts >>>> ::1 localhost ip6-localhost ip6-loopback >>>> fe00::0 ip6-localnet >>>> ff00::0 ip6-mcastprefix >>>> ff02::1 ip6-allnodes >>>> ff02::2 ip6-allrouters >>>> ff02::3 ip6-allhosts >>>> >>>> Thanks >>>> Ashish >>>> >>>> >>>> On Thu, Jun 23, 2011 at 1:40 AM, Edward J. Yoon <[email protected]> >>>> wrote: >>>>> What is your 'fs.default.name' of hadoop? >>>>> >>>>> What is your hostname? >>>>> >>>>> Have you seen the /etc/hosts file? >>>>> >>>>> On Thu, Jun 23, 2011 at 5:22 PM, Ashish Agarwal <[email protected]> >>>>> wrote: >>>>>> I already have made that change, but I still have the same error >>>>>> >>>>>> I have tried >>>>>> <value>localhost:40000</value> >>>>>> and also >>>>>> <value>local</value> >>>>>> >>>>>> >>>>>> Thanks >>>>>> Ashish >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Jun 23, 2011 at 1:12 AM, Edward J. Yoon <[email protected]> >>>>>> wrote: >>>>>>> >>>>>>> You need to set the 'bsp.master.address' to localhost or your hostname. >>>>>>> >>>>>>> conf/hama-site.xml: >>>>>>> >>>>>>> <property> >>>>>>> <name>bsp.master.address</name> >>>>>>> <value>localhost:40000</value> >>>>>>> </property> >>>>>>> >>>>>>> On Thu, Jun 23, 2011 at 5:01 PM, Ashish Agarwal >>>>>>> <[email protected]> wrote: >>>>>>> > Now, I am having a problem starting the bspmaster :( >>>>>>> > bin/hama bspmaster >>>>>>> > >>>>>>> > >>>>>>> > hadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$ bin/hama bspmaster >>>>>>> > 11/06/23 01:00:13 INFO hama.BSPMasterRunner: STARTUP_MSG: >>>>>>> > /************************************************************ >>>>>>> > STARTUP_MSG: Starting BSPMaster >>>>>>> > STARTUP_MSG: host = ubuntu/127.0.1.1 >>>>>>> > STARTUP_MSG: args = [] >>>>>>> > STARTUP_MSG: version = 0.20.2 >>>>>>> > STARTUP_MSG: build = -r ; compiled by 'chrisdo' on Wed Mar 10 >>>>>>> > 13:22:54 PST 2010 >>>>>>> > ************************************************************/ >>>>>>> > 11/06/23 01:00:13 INFO bsp.BSPMaster: RPC BSPMaster: host local port >>>>>>> > 40000 >>>>>>> > 11/06/23 01:00:14 FATAL hama.BSPMasterRunner: >>>>>>> > java.net.UnknownHostException: Invalid hostname for server: local >>>>>>> > at org.apache.hadoop.ipc.Server.bind(Server.java:198) >>>>>>> > at >>>>>>> > org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:253) >>>>>>> > at org.apache.hadoop.ipc.Server.<init>(Server.java:1026) >>>>>>> > at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:488) >>>>>>> > at org.apache.hadoop.ipc.RPC.getServer(RPC.java:450) >>>>>>> > at org.apache.hadoop.ipc.RPC.getServer(RPC.java:441) >>>>>>> > at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:248) >>>>>>> > at >>>>>>> > org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:419) >>>>>>> > at >>>>>>> > org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:413) >>>>>>> > at org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:46) >>>>>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >>>>>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >>>>>>> > at >>>>>>> > org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:56) >>>>>>> > >>>>>>> > 11/06/23 01:00:14 INFO hama.BSPMasterRunner: SHUTDOWN_MSG: >>>>>>> > /************************************************************ >>>>>>> > SHUTDOWN_MSG: Shutting down BSPMaster at ubuntu/127.0.1.1 >>>>>>> > ************************************************************/ >>>>>>> > hadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$ >>>>>>> > >>>>>>> > >>>>>>> > Thanks >>>>>>> > Ashish >>>>>>> > >>>>>>> > On Thu, Jun 23, 2011 at 12:49 AM, Edward J. Yoon >>>>>>> > <[email protected]> wrote: >>>>>>> >> If you want to execute a command in the background without watching >>>>>>> >> logs, you can do the following: >>>>>>> >> >>>>>>> >> % bin/hama zookeeper & >>>>>>> >> >>>>>>> >> On Thu, Jun 23, 2011 at 4:43 PM, Ashish Agarwal >>>>>>> >> <[email protected]> wrote: >>>>>>> >>> How long does it take to execute the command >>>>>>> >>> bin/hama zookeeper >>>>>>> >>> >>>>>>> >>> I had to stop the execution as it stopped doing anything after some >>>>>>> >>> time. >>>>>>> >>> >>>>>>> >>> hadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$ bin/hama >>>>>>> >>> zookeeper >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServerMain: Starting server >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 >>>>>>> >>> 17:14 >>>>>>> >>> GMT >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:host.name=ubuntu.ubuntu-domain >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:java.version=1.6.0_24 >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:java.vendor=Sun Microsystems Inc. >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.24/jre >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:java.class.path=/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../target/classes:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../hama-**.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/commons-cli-1.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/commons-logging-1.0.4.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/hadoop-core-0.20.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/hadoop-test-0.20.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/jetty-6.1H.22.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/jetty-annotations-6.1H.22.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/jetty-util-6.1H.22.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/junit-3.8.1.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/log4j-1.2.15.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/slf4j-log4j12-1.5.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/zookeeper-3.3.1.jar >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.24/jre/lib/i386/server:/usr/lib/jvm/java-6-sun-1.6.0.24/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.24/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:java.io.tmpdir=/tmp >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:java.compiler=<NA> >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:os.name=Linux >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:os.arch=i386 >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:os.version=2.6.35-28-generic >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:user.name=hadoop >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:user.home=/home/hadoop >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server >>>>>>> >>> environment:user.dir=/home/hadoop/Desktop/SecondWorkspace/hama-trunk >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: tickTime set to 3000 >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: minSessionTimeout >>>>>>> >>> set to -1 >>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: maxSessionTimeout >>>>>>> >>> set to 1200000 >>>>>>> >>> 11/06/23 00:27:41 INFO server.NIOServerCnxn: binding to port >>>>>>> >>> 0.0.0.0/0.0.0.0:21810 >>>>>>> >>> 11/06/23 00:27:41 INFO persistence.FileSnap: Reading snapshot >>>>>>> >>> /tmp/hama-hadoop/zookeeper/version-2/snapshot.0 >>>>>>> >>> 11/06/23 00:27:41 INFO persistence.FileSnap: Reading snapshot >>>>>>> >>> /tmp/hama-hadoop/zookeeper/version-2/snapshot.0 >>>>>>> >>> 11/06/23 00:27:41 INFO persistence.FileTxnSnapLog: Snapshotting: 0 >>>>>>> >>> ^Chadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$ >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> Thanks >>>>>>> >>> Ashish >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> On Wed, Jun 22, 2011 at 3:31 PM, Joseph Boyd >>>>>>> >>> <[email protected]> wrote: >>>>>>> >>>> >>>>>>> >>>> On Wed, Jun 22, 2011 at 3:15 PM, Edward J. Yoon >>>>>>> >>>> <[email protected]> wrote: >>>>>>> >>>> > Instead removing 206 line, why don't we add dependencies as >>>>>>> >>>> > below? >>>>>>> >>>> >>>>>>> >>>> I think removing line 206 is the right thing to do. Hama doesn't >>>>>>> >>>> depend on commons-cli, hadoop-core does, and maven will properly >>>>>>> >>>> pick >>>>>>> >>>> that up and include the jar if allowed. >>>>>>> >>>> >>>>>>> >>>> The fact that Hama doesn't depend on the several items below was >>>>>>> >>>> correctly detected by maven, in the report description of HAMA-402. >>>>>>> >>>> >>>>>>> >>>> You don't mention a particular problem with including the >>>>>>> >>>> transitive >>>>>>> >>>> dependencies, but if your problem is that it includes too many >>>>>>> >>>> items, >>>>>>> >>>> maven does have options to exclude a particular transitive >>>>>>> >>>> dependency >>>>>>> >>>> on a case-by-case basis that might be more appropriate to use. >>>>>>> >>>> >>>>>>> >>>> Including the dependencies below as you suggest puts Hama in the >>>>>>> >>>> place >>>>>>> >>>> of maintaining a transitive dependency like commons-cli in >>>>>>> >>>> lock-step >>>>>>> >>>> hadoop-core, where the actual dependency comes from. >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> This is all just my $.02 worth, I'm open to arguments in favor of >>>>>>> >>>> <excludeTransitive/>, and there is often more than one way to skin >>>>>>> >>>> a >>>>>>> >>>> cat with respect to dependency management. >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> ...joe >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> > <dependency> >>>>>>> >>>> > <groupId>org.mortbay.jetty</groupId> >>>>>>> >>>> > <artifactId>jetty-annotations</artifactId> >>>>>>> >>>> > <version>6.1H.22</version> >>>>>>> >>>> > </dependency> >>>>>>> >>>> > <dependency> >>>>>>> >>>> > <groupId>org.slf4j</groupId> >>>>>>> >>>> > <artifactId>slf4j-log4j12</artifactId> >>>>>>> >>>> > <version>1.5.2</version> >>>>>>> >>>> > </dependency> >>>>>>> >>>> > <dependency> >>>>>>> >>>> > <groupId>commons-cli</groupId> >>>>>>> >>>> > <artifactId>commons-cli</artifactId> >>>>>>> >>>> > <version>1.2</version> >>>>>>> >>>> > </dependency> >>>>>>> >>>> > >>>>>>> >>>> > Thanks. >>>>>>> >>>> > >>>>>>> >>>> > On Thu, Jun 23, 2011 at 7:00 AM, Edward J. Yoon >>>>>>> >>>> > <[email protected]> wrote: >>>>>>> >>>> >> Oh... sorry, you're right. >>>>>>> >>>> >> >>>>>>> >>>> >> It looks like a bug. We have to fix this. >>>>>>> >>>> >> >>>>>>> >>>> >> [INFO] Total time: 4:36.098s >>>>>>> >>>> >> [INFO] Finished at: Thu Jun 23 06:58:28 KST 2011 >>>>>>> >>>> >> [INFO] Final Memory: 20M/51M >>>>>>> >>>> >> [INFO] >>>>>>> >>>> >> ------------------------------------------------------------------------ >>>>>>> >>>> >> edward@test ~/workspace/hama-trunk $ ls lib/ >>>>>>> >>>> >> commons-logging-1.0.4.jar hadoop-test-0.20.2.jar >>>>>>> >>>> >> jetty-util-6.1H.22.jar log4j-1.2.15.jar >>>>>>> >>>> >> hadoop-core-0.20.2.jar jetty-6.1H.22.jar >>>>>>> >>>> >> junit-3.8.1.jar >>>>>>> >>>> >> zookeeper-3.3.1.jar >>>>>>> >>>> >> edward@tweetple ~/workspace/hama-trunk $ bin/hama zookeeper >>>>>>> >>>> >> -bash: bin/hama: Permission denied >>>>>>> >>>> >> edward@test ~/workspace/hama-trunk $ chmod 777 bin/** >>>>>>> >>>> >> edward@test ~/workspace/hama-trunk $ bin/hama zookeeper >>>>>>> >>>> >> Exception in thread "main" java.lang.NoClassDefFoundError: >>>>>>> >>>> >> org/apache/commons/cli/ParseException >>>>>>> >>>> >> at >>>>>>> >>>> >> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59) >>>>>>> >>>> >> at >>>>>>> >>>> >> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >>>>>>> >>>> >> at >>>>>>> >>>> >> org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41) >>>>>>> >>>> >> Caused by: java.lang.ClassNotFoundException: >>>>>>> >>>> >> org.apache.commons.cli.ParseException >>>>>>> >>>> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:200) >>>>>>> >>>> >> at java.security.AccessController.doPrivileged(Native >>>>>>> >>>> >> Method) >>>>>>> >>>> >> at >>>>>>> >>>> >> java.net.URLClassLoader.findClass(URLClassLoader.java:188) >>>>>>> >>>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:303) >>>>>>> >>>> >> at >>>>>>> >>>> >> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>>>>> >>>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >>>>>>> >>>> >> at >>>>>>> >>>> >> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316) >>>>>>> >>>> >> ... 3 more >>>>>>> >>>> >> >>>>>>> >>>> >> >>>>>>> >>>> >> >>>>>>> >>>> >> On Thu, Jun 23, 2011 at 6:45 AM, Edward J. Yoon >>>>>>> >>>> >> <[email protected]> wrote: >>>>>>> >>>> >>> That's so weird. >>>>>>> >>>> >>> >>>>>>> >>>> >>> It works well on my test cluster. >>>>>>> >>>> >>> >>>>>>> >>>> >>> edward@slave:~/workspace/hama-trunk$ ls lib/ >>>>>>> >>>> >>> commons-logging-1.0.4.jar hadoop-test-0.20.2.jar >>>>>>> >>>> >>> jetty-util-6.1H.22.jar log4j-1.2.15.jar >>>>>>> >>>> >>> hadoop-core-0.20.2.jar jetty-6.1H.22.jar >>>>>>> >>>> >>> junit-3.8.1.jar >>>>>>> >>>> >>> zookeeper-3.3.1.jar >>>>>>> >>>> >>> edward@slave:~/workspace/hama-trunk$ bin/hama jar >>>>>>> >>>> >>> hama-examples-0.3.0-incubating-SNAPSHOT.jar pi >>>>>>> >>>> >>> 11/06/23 06:41:55 DEBUG bsp.BSPJobClient: >>>>>>> >>>> >>> BSPJobClient.submitJobDir: >>>>>>> >>>> >>> hdfs://slave.udanax.org:9000/tmp/hadoop-edward/bsp/system/submit_ebr5do >>>>>>> >>>> >>> 11/06/23 06:41:57 INFO bsp.BSPJobClient: Running job: >>>>>>> >>>> >>> job_201106230640_0001 >>>>>>> >>>> >>> 11/06/23 06:42:00 INFO bsp.BSPJobClient: Current supersteps >>>>>>> >>>> >>> number: 0 >>>>>>> >>>> >>> 11/06/23 06:42:03 INFO bsp.BSPJobClient: Current supersteps >>>>>>> >>>> >>> number: 1 >>>>>>> >>>> >>> 11/06/23 06:42:03 INFO bsp.BSPJobClient: The total number of >>>>>>> >>>> >>> supersteps: 1 >>>>>>> >>>> >>> Estimated value of PI is 3.1534 >>>>>>> >>>> >>> Job Finished in 7.886 seconds >>>>>>> >>>> >>> edward@slave:~/workspace/hama-trunk$ >>>>>>> >>>> >>> >>>>>>> >>>> >>> On Thu, Jun 23, 2011 at 4:21 AM, Joseph Boyd >>>>>>> >>>> >>> <[email protected]> wrote: >>>>>>> >>>> >>>> Replying to my own post ... >>>>>>> >>>> >>>> >>>>>>> >>>> >>>> On Wed, Jun 22, 2011 at 11:05 AM, Joseph Boyd >>>>>>> >>>> >>>> <[email protected]> wrote: >>>>>>> >>>> >>>>> On Thu, Jun 16, 2011 at 4:37 AM, Ashish Agarwal >>>>>>> >>>> >>>>> <[email protected]> wrote: >>>>>>> >>>> >>>>>> >>>>>>> >>>> >>>>>> When I try to run the hama daemons using the command - >>>>>>> >>>> >>>>>> bin/hama zookeeper >>>>>>> >>>> >>>>>> >>>>>>> >>>> >>>>>> I get this error - >>>>>>> >>>> >>>> >>>>>>> >>>> >>>> It looks like the problem is actually this line in pom.xml >>>>>>> >>>> >>>> (line 205) : >>>>>>> >>>> >>>> <excludeTransitive>true</excludeTransitive> >>>>>>> >>>> >>>> >>>>>>> >>>> >>>> This prevents commons-cli (and other transitive dependencies) >>>>>>> >>>> >>>> from >>>>>>> >>>> >>>> being copied to the lib dir like they should be. >>>>>>> >>>> >>>> >>>>>>> >>>> >>>> Removing line 205 fixes the NoClassDefFoundError for me. I >>>>>>> >>>> >>>> can open a >>>>>>> >>>> >>>> JIRA for this after lunch, the command line is fairly broken >>>>>>> >>>> >>>> with this >>>>>>> >>>> >>>> line in, especially after several dependencies were >>>>>>> >>>> >>>> (correctly) >>>>>>> >>>> >>>> removed in HAMA-402. >>>>>>> >>>> >>>> >>>>>>> >>>> >>>> >>>>>>> >>>> >>>> ...joe >>>>>>> >>>> >>>> >>>>>>> >>>> >>>> >>>>>>> >>>> >>>>> I'm getting the same NoClassDefFoundError using the svn >>>>>>> >>>> >>>>> code. The >>>>>>> >>>> >>>>> commons-cli dependency was removed recently as part of >>>>>>> >>>> >>>>> HAMA-402, and >>>>>>> >>>> >>>>> re-adding it fixes the NoClassDefFoundError. >>>>>>> >>>> >>>>> >>>>>>> >>>> >>>>> I believe commons-cli should have been pulled as part of the >>>>>>> >>>> >>>>> hadoop-core dependency, and wasn't. (though my maven skills >>>>>>> >>>> >>>>> are >>>>>>> >>>> >>>>> rusty, and I'm no longer sure just how transitive >>>>>>> >>>> >>>>> dependencies are >>>>>>> >>>> >>>>> pulled) >>>>>>> >>>> >>>>> >>>>>>> >>>> >>>>> While a quick work-around is to re-add the dependency here, >>>>>>> >>>> >>>>> the proper >>>>>>> >>>> >>>>> fix is probably to fix the hadoop-core pom, and properly >>>>>>> >>>> >>>>> mark that it >>>>>>> >>>> >>>>> depends on commons-cli. >>>>>>> >>>> >>>>> >>>>>>> >>>> >>>>> >>>>>>> >>>> >>>>> ...joe >>>>>>> >>>> >>>>> >>>>>>> >>>> >>>>> >>>>>>> >>>> >>>>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>>>>> >>>> >>>>>> org/apache/commons/cli/ParseException >>>>>>> >>>> >>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59) >>>>>>> >>>> >>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >>>>>>> >>>> >>>>>> at >>>>>>> >>>> >>>>>> org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41) >>>>>>> >>>> >>>>>> Caused by: java.lang.ClassNotFoundException: >>>>>>> >>>> >>>>>> org.apache.commons.cli.ParseException >>>>>>> >>>> >>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>>>>>> >>>> >>>>>> at java.security.AccessController.doPrivileged(Native >>>>>>> >>>> >>>>>> Method) >>>>>>> >>>> >>>>>> at >>>>>>> >>>> >>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>>>>>> >>>> >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >>>>>>> >>>> >>>>>> at >>>>>>> >>>> >>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>>>>> >>>> >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >>>>>>> >>>> >>>>>> ... 3 more >>>>>>> >>>> >>>>>> >>>>>>> >>>> >>>>>> >>>>>>> >>>> >>>>>> I tried installing it a few times, but can't think of a >>>>>>> >>>> >>>>>> solution to the >>>>>>> >>>> >>>>>> problem. Can you please help ? >>>>>>> >>>> >>>>>> >>>>>>> >>>> >>>>>> Thank you >>>>>>> >>>> >>>>>> >>>>>>> >>>> >>>>>> Sincerely, >>>>>>> >>>> >>>>>> Ashish >>>>>>> >>>> >>>>>> >>>>>>> >>>> >>>>>> >>>>>>> >>>> >>>>>> On Wed, Jun 15, 2011 at 3:18 AM, Edward J. Yoon >>>>>>> >>>> >>>>>> <[email protected]>wrote: >>>>>>> >>>> >>>>>> >>>>>>> >>>> >>>>>>> Hi, >>>>>>> >>>> >>>>>>> >>>>>>> >>>> >>>>>>> I would suggest following steps to debug: >>>>>>> >>>> >>>>>>> >>>>>>> >>>> >>>>>>> 1. Install the Hadoop 0.20.x on your laptop. >>>>>>> >>>> >>>>>>> 2. Run the Hama deamons with 3 terminals. >>>>>>> >>>> >>>>>>> >>>>>>> >>>> >>>>>>> % bin/hama zookeeper >>>>>>> >>>> >>>>>>> % bin/hama bspmaster >>>>>>> >>>> >>>>>>> % bin/hama groom >>>>>>> >>>> >>>>>>> >>>>>>> >>>> >>>>>>> 3. Use the logging API to log debug. >>>>>>> >>>> >>>>>>> For example, LOG.debug("like this"); >>>>>>> >>>> >>>>>>> >>>>>>> >>>> >>>>>>> On Wed, Jun 15, 2011 at 6:48 PM, Ashish Agarwal >>>>>>> >>>> >>>>>>> <[email protected]> >>>>>>> >>>> >>>>>>> wrote: >>>>>>> >>>> >>>>>>> > Hi, >>>>>>> >>>> >>>>>>> > >>>>>>> >>>> >>>>>>> > I was wondering if anyone can give me instructions on >>>>>>> >>>> >>>>>>> > how to debug code >>>>>>> >>>> >>>>>>> in >>>>>>> >>>> >>>>>>> > distributed or pseudo-distributed mode ? It would really >>>>>>> >>>> >>>>>>> > help me. >>>>>>> >>>> >>>>>>> > >>>>>>> >>>> >>>>>>> > Thank you >>>>>>> >>>> >>>>>>> > >>>>>>> >>>> >>>>>>> > Sincerely, >>>>>>> >>>> >>>>>>> > Ashish >>>>>>> >>>> >>>>>>> > >>>>>>> >>>> >>>>>>> > >>>>>>> >>>> >>>>>>> > >>>>>>> >>>> >>>>>>> > On Sat, Jun 11, 2011 at 3:41 AM, Ashish Agarwal >>>>>>> >>>> >>>>>>> > <[email protected] >>>>>>> >>>> >>>>>>> >wrote: >>>>>>> >>>> >>>>>>> > >>>>>>> >>>> >>>>>>> >> >>>>>>> >>>> >>>>>>> >> Hi, >>>>>>> >>>> >>>>>>> >> >>>>>>> >>>> >>>>>>> >> I have written some code for my GSoC task (Hama-367) >>>>>>> >>>> >>>>>>> >> and am now trying >>>>>>> >>>> >>>>>>> to >>>>>>> >>>> >>>>>>> >> debug. >>>>>>> >>>> >>>>>>> >> >>>>>>> >>>> >>>>>>> >> I am trying to set up the environment into >>>>>>> >>>> >>>>>>> >> pseudo-distributed mode so I >>>>>>> >>>> >>>>>>> can >>>>>>> >>>> >>>>>>> >> debug. I think the value of bsp.master.address in >>>>>>> >>>> >>>>>>> >> hama-site.xml has to >>>>>>> >>>> >>>>>>> be >>>>>>> >>>> >>>>>>> >> local as this means LocalBSPRunner (as localhost would >>>>>>> >>>> >>>>>>> >> call 127.0.0.1 >>>>>>> >>>> >>>>>>> >> server), but where do I specify that in the code in >>>>>>> >>>> >>>>>>> >> eclipse ? >>>>>>> >>>> >>>>>>> >> >>>>>>> >>>> >>>>>>> >> My next question is - what is the difference between >>>>>>> >>>> >>>>>>> >> MiniBSPCluster and >>>>>>> >>>> >>>>>>> >> LocalBSPRunner and which file should I be using to be >>>>>>> >>>> >>>>>>> >> able to step >>>>>>> >>>> >>>>>>> through >>>>>>> >>>> >>>>>>> >> the code ? >>>>>>> >>>> >>>>>>> >> >>>>>>> >>>> >>>>>>> >> These are the processes currently running on my machine >>>>>>> >>>> >>>>>>> >> - >>>>>>> >>>> >>>>>>> >> >>>>>>> >>>> >>>>>>> >> 6112 NameNode >>>>>>> >>>> >>>>>>> >> 3319 ZooKeeperRunner >>>>>>> >>>> >>>>>>> >> 3018 >>>>>>> >>>> >>>>>>> >> 6449 SecondaryNameNode >>>>>>> >>>> >>>>>>> >> 5709 HMaster >>>>>>> >>>> >>>>>>> >> 5643 HQuorumPeer >>>>>>> >>>> >>>>>>> >> 6684 TaskTracker >>>>>>> >>>> >>>>>>> >> 6526 JobTracker >>>>>>> >>>> >>>>>>> >> 8074 Jps >>>>>>> >>>> >>>>>>> >> 6277 DataNode >>>>>>> >>>> >>>>>>> >> >>>>>>> >>>> >>>>>>> >> Thank you, >>>>>>> >>>> >>>>>>> >> Ashish >>>>>>> >>>> >>>>>>> >> >>>>>>> >>>> >>>>>>> > >>>>>>> >>>> >>>>>>> >>>>>>> >>>> >>>>>>> >>>>>>> >>>> >>>>>>> >>>>>>> >>>> >>>>>>> -- >>>>>>> >>>> >>>>>>> Best Regards, Edward J. Yoon >>>>>>> >>>> >>>>>>> @eddieyoon >>>>>>> >>>> >>>>>>> >>>>>>> >>>> >>>>>> >>>>>>> >>>> >>>>> >>>>>>> >>>> >>>> >>>>>>> >>>> >>> >>>>>>> >>>> >>> >>>>>>> >>>> >>> >>>>>>> >>>> >>> -- >>>>>>> >>>> >>> Best Regards, Edward J. Yoon >>>>>>> >>>> >>> @eddieyoon >>>>>>> >>>> >>> >>>>>>> >>>> >> >>>>>>> >>>> >> >>>>>>> >>>> >> >>>>>>> >>>> >> -- >>>>>>> >>>> >> Best Regards, Edward J. Yoon >>>>>>> >>>> >> @eddieyoon >>>>>>> >>>> >> >>>>>>> >>>> > >>>>>>> >>>> > >>>>>>> >>>> > >>>>>>> >>>> > -- >>>>>>> >>>> > Best Regards, Edward J. Yoon >>>>>>> >>>> > @eddieyoon >>>>>>> >>>> > >>>>>>> >>> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> -- >>>>>>> >> Best Regards, Edward J. Yoon >>>>>>> >> @eddieyoon >>>>>>> >> >>>>>>> > >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Best Regards, Edward J. Yoon >>>>>>> @eddieyoon >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best Regards, Edward J. Yoon >>>>> @eddieyoon >>>>> >>>> >>> >>> >>> >>> -- >>> Best Regards, Edward J. Yoon >>> @eddieyoon >>> >> > > > > -- > Best Regards, Edward J. Yoon > @eddieyoon >
