I already have made that change, but I still have the same error I have tried <value>localhost:40000</value> and also <value>local</value>
Thanks Ashish On Thu, Jun 23, 2011 at 1:12 AM, Edward J. Yoon <[email protected]> wrote: > > You need to set the 'bsp.master.address' to localhost or your hostname. > > conf/hama-site.xml: > > <property> > <name>bsp.master.address</name> > <value>localhost:40000</value> > </property> > > On Thu, Jun 23, 2011 at 5:01 PM, Ashish Agarwal <[email protected]> > wrote: > > Now, I am having a problem starting the bspmaster :( > > bin/hama bspmaster > > > > > > hadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$ bin/hama bspmaster > > 11/06/23 01:00:13 INFO hama.BSPMasterRunner: STARTUP_MSG: > > /************************************************************ > > STARTUP_MSG: Starting BSPMaster > > STARTUP_MSG: host = ubuntu/127.0.1.1 > > STARTUP_MSG: args = [] > > STARTUP_MSG: version = 0.20.2 > > STARTUP_MSG: build = -r ; compiled by 'chrisdo' on Wed Mar 10 > > 13:22:54 PST 2010 > > ************************************************************/ > > 11/06/23 01:00:13 INFO bsp.BSPMaster: RPC BSPMaster: host local port 40000 > > 11/06/23 01:00:14 FATAL hama.BSPMasterRunner: > > java.net.UnknownHostException: Invalid hostname for server: local > > at org.apache.hadoop.ipc.Server.bind(Server.java:198) > > at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:253) > > at org.apache.hadoop.ipc.Server.<init>(Server.java:1026) > > at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:488) > > at org.apache.hadoop.ipc.RPC.getServer(RPC.java:450) > > at org.apache.hadoop.ipc.RPC.getServer(RPC.java:441) > > at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:248) > > at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:419) > > at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:413) > > at org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:46) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > > at org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:56) > > > > 11/06/23 01:00:14 INFO hama.BSPMasterRunner: SHUTDOWN_MSG: > > /************************************************************ > > SHUTDOWN_MSG: Shutting down BSPMaster at ubuntu/127.0.1.1 > > ************************************************************/ > > hadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$ > > > > > > Thanks > > Ashish > > > > On Thu, Jun 23, 2011 at 12:49 AM, Edward J. Yoon <[email protected]> > > wrote: > >> If you want to execute a command in the background without watching > >> logs, you can do the following: > >> > >> % bin/hama zookeeper & > >> > >> On Thu, Jun 23, 2011 at 4:43 PM, Ashish Agarwal <[email protected]> > >> wrote: > >>> How long does it take to execute the command > >>> bin/hama zookeeper > >>> > >>> I had to stop the execution as it stopped doing anything after some time. > >>> > >>> hadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$ bin/hama zookeeper > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServerMain: Starting server > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14 > >>> GMT > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:host.name=ubuntu.ubuntu-domain > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:java.version=1.6.0_24 > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:java.vendor=Sun Microsystems Inc. > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.24/jre > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:java.class.path=/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../target/classes:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../hama-**.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/commons-cli-1.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/commons-logging-1.0.4.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/hadoop-core-0.20.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/hadoop-test-0.20.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/jetty-6.1H.22.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/jetty-annotations-6.1H.22.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/jetty-util-6.1H.22.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/junit-3.8.1.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/log4j-1.2.15.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/slf4j-log4j12-1.5.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/zookeeper-3.3.1.jar > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.24/jre/lib/i386/server:/usr/lib/jvm/java-6-sun-1.6.0.24/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.24/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:java.io.tmpdir=/tmp > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:java.compiler=<NA> > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:os.name=Linux > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:os.arch=i386 > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:os.version=2.6.35-28-generic > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:user.name=hadoop > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:user.home=/home/hadoop > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server > >>> environment:user.dir=/home/hadoop/Desktop/SecondWorkspace/hama-trunk > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: tickTime set to 3000 > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: minSessionTimeout set to -1 > >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: maxSessionTimeout set to > >>> 1200000 > >>> 11/06/23 00:27:41 INFO server.NIOServerCnxn: binding to port > >>> 0.0.0.0/0.0.0.0:21810 > >>> 11/06/23 00:27:41 INFO persistence.FileSnap: Reading snapshot > >>> /tmp/hama-hadoop/zookeeper/version-2/snapshot.0 > >>> 11/06/23 00:27:41 INFO persistence.FileSnap: Reading snapshot > >>> /tmp/hama-hadoop/zookeeper/version-2/snapshot.0 > >>> 11/06/23 00:27:41 INFO persistence.FileTxnSnapLog: Snapshotting: 0 > >>> ^Chadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$ > >>> > >>> > >>> Thanks > >>> Ashish > >>> > >>> > >>> > >>> On Wed, Jun 22, 2011 at 3:31 PM, Joseph Boyd > >>> <[email protected]> wrote: > >>>> > >>>> On Wed, Jun 22, 2011 at 3:15 PM, Edward J. Yoon <[email protected]> > >>>> wrote: > >>>> > Instead removing 206 line, why don't we add dependencies as below? > >>>> > >>>> I think removing line 206 is the right thing to do. Hama doesn't > >>>> depend on commons-cli, hadoop-core does, and maven will properly pick > >>>> that up and include the jar if allowed. > >>>> > >>>> The fact that Hama doesn't depend on the several items below was > >>>> correctly detected by maven, in the report description of HAMA-402. > >>>> > >>>> You don't mention a particular problem with including the transitive > >>>> dependencies, but if your problem is that it includes too many items, > >>>> maven does have options to exclude a particular transitive dependency > >>>> on a case-by-case basis that might be more appropriate to use. > >>>> > >>>> Including the dependencies below as you suggest puts Hama in the place > >>>> of maintaining a transitive dependency like commons-cli in lock-step > >>>> hadoop-core, where the actual dependency comes from. > >>>> > >>>> > >>>> This is all just my $.02 worth, I'm open to arguments in favor of > >>>> <excludeTransitive/>, and there is often more than one way to skin a > >>>> cat with respect to dependency management. > >>>> > >>>> > >>>> ...joe > >>>> > >>>> > >>>> > >>>> > <dependency> > >>>> > <groupId>org.mortbay.jetty</groupId> > >>>> > <artifactId>jetty-annotations</artifactId> > >>>> > <version>6.1H.22</version> > >>>> > </dependency> > >>>> > <dependency> > >>>> > <groupId>org.slf4j</groupId> > >>>> > <artifactId>slf4j-log4j12</artifactId> > >>>> > <version>1.5.2</version> > >>>> > </dependency> > >>>> > <dependency> > >>>> > <groupId>commons-cli</groupId> > >>>> > <artifactId>commons-cli</artifactId> > >>>> > <version>1.2</version> > >>>> > </dependency> > >>>> > > >>>> > Thanks. > >>>> > > >>>> > On Thu, Jun 23, 2011 at 7:00 AM, Edward J. Yoon > >>>> > <[email protected]> wrote: > >>>> >> Oh... sorry, you're right. > >>>> >> > >>>> >> It looks like a bug. We have to fix this. > >>>> >> > >>>> >> [INFO] Total time: 4:36.098s > >>>> >> [INFO] Finished at: Thu Jun 23 06:58:28 KST 2011 > >>>> >> [INFO] Final Memory: 20M/51M > >>>> >> [INFO] > >>>> >> ------------------------------------------------------------------------ > >>>> >> edward@test ~/workspace/hama-trunk $ ls lib/ > >>>> >> commons-logging-1.0.4.jar hadoop-test-0.20.2.jar > >>>> >> jetty-util-6.1H.22.jar log4j-1.2.15.jar > >>>> >> hadoop-core-0.20.2.jar jetty-6.1H.22.jar junit-3.8.1.jar > >>>> >> zookeeper-3.3.1.jar > >>>> >> edward@tweetple ~/workspace/hama-trunk $ bin/hama zookeeper > >>>> >> -bash: bin/hama: Permission denied > >>>> >> edward@test ~/workspace/hama-trunk $ chmod 777 bin/** > >>>> >> edward@test ~/workspace/hama-trunk $ bin/hama zookeeper > >>>> >> Exception in thread "main" java.lang.NoClassDefFoundError: > >>>> >> org/apache/commons/cli/ParseException > >>>> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59) > >>>> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > >>>> >> at > >>>> >> org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41) > >>>> >> Caused by: java.lang.ClassNotFoundException: > >>>> >> org.apache.commons.cli.ParseException > >>>> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > >>>> >> at java.security.AccessController.doPrivileged(Native Method) > >>>> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > >>>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:303) > >>>> >> at > >>>> >> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > >>>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > >>>> >> at > >>>> >> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316) > >>>> >> ... 3 more > >>>> >> > >>>> >> > >>>> >> > >>>> >> On Thu, Jun 23, 2011 at 6:45 AM, Edward J. Yoon > >>>> >> <[email protected]> wrote: > >>>> >>> That's so weird. > >>>> >>> > >>>> >>> It works well on my test cluster. > >>>> >>> > >>>> >>> edward@slave:~/workspace/hama-trunk$ ls lib/ > >>>> >>> commons-logging-1.0.4.jar hadoop-test-0.20.2.jar > >>>> >>> jetty-util-6.1H.22.jar log4j-1.2.15.jar > >>>> >>> hadoop-core-0.20.2.jar jetty-6.1H.22.jar junit-3.8.1.jar > >>>> >>> zookeeper-3.3.1.jar > >>>> >>> edward@slave:~/workspace/hama-trunk$ bin/hama jar > >>>> >>> hama-examples-0.3.0-incubating-SNAPSHOT.jar pi > >>>> >>> 11/06/23 06:41:55 DEBUG bsp.BSPJobClient: BSPJobClient.submitJobDir: > >>>> >>> hdfs://slave.udanax.org:9000/tmp/hadoop-edward/bsp/system/submit_ebr5do > >>>> >>> 11/06/23 06:41:57 INFO bsp.BSPJobClient: Running job: > >>>> >>> job_201106230640_0001 > >>>> >>> 11/06/23 06:42:00 INFO bsp.BSPJobClient: Current supersteps number: 0 > >>>> >>> 11/06/23 06:42:03 INFO bsp.BSPJobClient: Current supersteps number: 1 > >>>> >>> 11/06/23 06:42:03 INFO bsp.BSPJobClient: The total number of > >>>> >>> supersteps: 1 > >>>> >>> Estimated value of PI is 3.1534 > >>>> >>> Job Finished in 7.886 seconds > >>>> >>> edward@slave:~/workspace/hama-trunk$ > >>>> >>> > >>>> >>> On Thu, Jun 23, 2011 at 4:21 AM, Joseph Boyd > >>>> >>> <[email protected]> wrote: > >>>> >>>> Replying to my own post ... > >>>> >>>> > >>>> >>>> On Wed, Jun 22, 2011 at 11:05 AM, Joseph Boyd > >>>> >>>> <[email protected]> wrote: > >>>> >>>>> On Thu, Jun 16, 2011 at 4:37 AM, Ashish Agarwal > >>>> >>>>> <[email protected]> wrote: > >>>> >>>>>> > >>>> >>>>>> When I try to run the hama daemons using the command - > >>>> >>>>>> bin/hama zookeeper > >>>> >>>>>> > >>>> >>>>>> I get this error - > >>>> >>>> > >>>> >>>> It looks like the problem is actually this line in pom.xml (line > >>>> >>>> 205) : > >>>> >>>> <excludeTransitive>true</excludeTransitive> > >>>> >>>> > >>>> >>>> This prevents commons-cli (and other transitive dependencies) from > >>>> >>>> being copied to the lib dir like they should be. > >>>> >>>> > >>>> >>>> Removing line 205 fixes the NoClassDefFoundError for me. I can > >>>> >>>> open a > >>>> >>>> JIRA for this after lunch, the command line is fairly broken with > >>>> >>>> this > >>>> >>>> line in, especially after several dependencies were (correctly) > >>>> >>>> removed in HAMA-402. > >>>> >>>> > >>>> >>>> > >>>> >>>> ...joe > >>>> >>>> > >>>> >>>> > >>>> >>>>> I'm getting the same NoClassDefFoundError using the svn code. The > >>>> >>>>> commons-cli dependency was removed recently as part of HAMA-402, > >>>> >>>>> and > >>>> >>>>> re-adding it fixes the NoClassDefFoundError. > >>>> >>>>> > >>>> >>>>> I believe commons-cli should have been pulled as part of the > >>>> >>>>> hadoop-core dependency, and wasn't. (though my maven skills are > >>>> >>>>> rusty, and I'm no longer sure just how transitive dependencies are > >>>> >>>>> pulled) > >>>> >>>>> > >>>> >>>>> While a quick work-around is to re-add the dependency here, the > >>>> >>>>> proper > >>>> >>>>> fix is probably to fix the hadoop-core pom, and properly mark that > >>>> >>>>> it > >>>> >>>>> depends on commons-cli. > >>>> >>>>> > >>>> >>>>> > >>>> >>>>> ...joe > >>>> >>>>> > >>>> >>>>> > >>>> >>>>>> Exception in thread "main" java.lang.NoClassDefFoundError: > >>>> >>>>>> org/apache/commons/cli/ParseException > >>>> >>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59) > >>>> >>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > >>>> >>>>>> at org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41) > >>>> >>>>>> Caused by: java.lang.ClassNotFoundException: > >>>> >>>>>> org.apache.commons.cli.ParseException > >>>> >>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > >>>> >>>>>> at java.security.AccessController.doPrivileged(Native Method) > >>>> >>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > >>>> >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > >>>> >>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > >>>> >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > >>>> >>>>>> ... 3 more > >>>> >>>>>> > >>>> >>>>>> > >>>> >>>>>> I tried installing it a few times, but can't think of a solution > >>>> >>>>>> to the > >>>> >>>>>> problem. Can you please help ? > >>>> >>>>>> > >>>> >>>>>> Thank you > >>>> >>>>>> > >>>> >>>>>> Sincerely, > >>>> >>>>>> Ashish > >>>> >>>>>> > >>>> >>>>>> > >>>> >>>>>> On Wed, Jun 15, 2011 at 3:18 AM, Edward J. Yoon > >>>> >>>>>> <[email protected]>wrote: > >>>> >>>>>> > >>>> >>>>>>> Hi, > >>>> >>>>>>> > >>>> >>>>>>> I would suggest following steps to debug: > >>>> >>>>>>> > >>>> >>>>>>> 1. Install the Hadoop 0.20.x on your laptop. > >>>> >>>>>>> 2. Run the Hama deamons with 3 terminals. > >>>> >>>>>>> > >>>> >>>>>>> % bin/hama zookeeper > >>>> >>>>>>> % bin/hama bspmaster > >>>> >>>>>>> % bin/hama groom > >>>> >>>>>>> > >>>> >>>>>>> 3. Use the logging API to log debug. > >>>> >>>>>>> For example, LOG.debug("like this"); > >>>> >>>>>>> > >>>> >>>>>>> On Wed, Jun 15, 2011 at 6:48 PM, Ashish Agarwal > >>>> >>>>>>> <[email protected]> > >>>> >>>>>>> wrote: > >>>> >>>>>>> > Hi, > >>>> >>>>>>> > > >>>> >>>>>>> > I was wondering if anyone can give me instructions on how to > >>>> >>>>>>> > debug code > >>>> >>>>>>> in > >>>> >>>>>>> > distributed or pseudo-distributed mode ? It would really help > >>>> >>>>>>> > me. > >>>> >>>>>>> > > >>>> >>>>>>> > Thank you > >>>> >>>>>>> > > >>>> >>>>>>> > Sincerely, > >>>> >>>>>>> > Ashish > >>>> >>>>>>> > > >>>> >>>>>>> > > >>>> >>>>>>> > > >>>> >>>>>>> > On Sat, Jun 11, 2011 at 3:41 AM, Ashish Agarwal > >>>> >>>>>>> > <[email protected] > >>>> >>>>>>> >wrote: > >>>> >>>>>>> > > >>>> >>>>>>> >> > >>>> >>>>>>> >> Hi, > >>>> >>>>>>> >> > >>>> >>>>>>> >> I have written some code for my GSoC task (Hama-367) and am > >>>> >>>>>>> >> now trying > >>>> >>>>>>> to > >>>> >>>>>>> >> debug. > >>>> >>>>>>> >> > >>>> >>>>>>> >> I am trying to set up the environment into pseudo-distributed > >>>> >>>>>>> >> mode so I > >>>> >>>>>>> can > >>>> >>>>>>> >> debug. I think the value of bsp.master.address in > >>>> >>>>>>> >> hama-site.xml has to > >>>> >>>>>>> be > >>>> >>>>>>> >> local as this means LocalBSPRunner (as localhost would call > >>>> >>>>>>> >> 127.0.0.1 > >>>> >>>>>>> >> server), but where do I specify that in the code in eclipse ? > >>>> >>>>>>> >> > >>>> >>>>>>> >> My next question is - what is the difference between > >>>> >>>>>>> >> MiniBSPCluster and > >>>> >>>>>>> >> LocalBSPRunner and which file should I be using to be able > >>>> >>>>>>> >> to step > >>>> >>>>>>> through > >>>> >>>>>>> >> the code ? > >>>> >>>>>>> >> > >>>> >>>>>>> >> These are the processes currently running on my machine - > >>>> >>>>>>> >> > >>>> >>>>>>> >> 6112 NameNode > >>>> >>>>>>> >> 3319 ZooKeeperRunner > >>>> >>>>>>> >> 3018 > >>>> >>>>>>> >> 6449 SecondaryNameNode > >>>> >>>>>>> >> 5709 HMaster > >>>> >>>>>>> >> 5643 HQuorumPeer > >>>> >>>>>>> >> 6684 TaskTracker > >>>> >>>>>>> >> 6526 JobTracker > >>>> >>>>>>> >> 8074 Jps > >>>> >>>>>>> >> 6277 DataNode > >>>> >>>>>>> >> > >>>> >>>>>>> >> Thank you, > >>>> >>>>>>> >> Ashish > >>>> >>>>>>> >> > >>>> >>>>>>> > > >>>> >>>>>>> > >>>> >>>>>>> > >>>> >>>>>>> > >>>> >>>>>>> -- > >>>> >>>>>>> Best Regards, Edward J. Yoon > >>>> >>>>>>> @eddieyoon > >>>> >>>>>>> > >>>> >>>>>> > >>>> >>>>> > >>>> >>>> > >>>> >>> > >>>> >>> > >>>> >>> > >>>> >>> -- > >>>> >>> Best Regards, Edward J. Yoon > >>>> >>> @eddieyoon > >>>> >>> > >>>> >> > >>>> >> > >>>> >> > >>>> >> -- > >>>> >> Best Regards, Edward J. Yoon > >>>> >> @eddieyoon > >>>> >> > >>>> > > >>>> > > >>>> > > >>>> > -- > >>>> > Best Regards, Edward J. Yoon > >>>> > @eddieyoon > >>>> > > >>> > >> > >> > >> > >> -- > >> Best Regards, Edward J. Yoon > >> @eddieyoon > >> > > > > > > -- > Best Regards, Edward J. Yoon > @eddieyoon
