How long does it take to execute the command bin/hama zookeeper I had to stop the execution as it stopped doing anything after some time.
hadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$ bin/hama zookeeper 11/06/23 00:27:41 INFO server.ZooKeeperServerMain: Starting server 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14 GMT 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:host.name=ubuntu.ubuntu-domain 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:java.version=1.6.0_24 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:java.vendor=Sun Microsystems Inc. 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.24/jre 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:java.class.path=/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../target/classes:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../hama-**.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/commons-cli-1.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/commons-logging-1.0.4.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/hadoop-core-0.20.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/hadoop-test-0.20.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/jetty-6.1H.22.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/jetty-annotations-6.1H.22.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/jetty-util-6.1H.22.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/junit-3.8.1.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/log4j-1.2.15.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/slf4j-log4j12-1.5.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/zookeeper-3.3.1.jar 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.24/jre/lib/i386/server:/usr/lib/jvm/java-6-sun-1.6.0.24/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.24/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:java.io.tmpdir=/tmp 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:java.compiler=<NA> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:os.name=Linux 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:os.arch=i386 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:os.version=2.6.35-28-generic 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:user.name=hadoop 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:user.home=/home/hadoop 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server environment:user.dir=/home/hadoop/Desktop/SecondWorkspace/hama-trunk 11/06/23 00:27:41 INFO server.ZooKeeperServer: tickTime set to 3000 11/06/23 00:27:41 INFO server.ZooKeeperServer: minSessionTimeout set to -1 11/06/23 00:27:41 INFO server.ZooKeeperServer: maxSessionTimeout set to 1200000 11/06/23 00:27:41 INFO server.NIOServerCnxn: binding to port 0.0.0.0/0.0.0.0:21810 11/06/23 00:27:41 INFO persistence.FileSnap: Reading snapshot /tmp/hama-hadoop/zookeeper/version-2/snapshot.0 11/06/23 00:27:41 INFO persistence.FileSnap: Reading snapshot /tmp/hama-hadoop/zookeeper/version-2/snapshot.0 11/06/23 00:27:41 INFO persistence.FileTxnSnapLog: Snapshotting: 0 ^Chadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$ Thanks Ashish On Wed, Jun 22, 2011 at 3:31 PM, Joseph Boyd <[email protected]> wrote: > > On Wed, Jun 22, 2011 at 3:15 PM, Edward J. Yoon <[email protected]> wrote: > > Instead removing 206 line, why don't we add dependencies as below? > > I think removing line 206 is the right thing to do. Hama doesn't > depend on commons-cli, hadoop-core does, and maven will properly pick > that up and include the jar if allowed. > > The fact that Hama doesn't depend on the several items below was > correctly detected by maven, in the report description of HAMA-402. > > You don't mention a particular problem with including the transitive > dependencies, but if your problem is that it includes too many items, > maven does have options to exclude a particular transitive dependency > on a case-by-case basis that might be more appropriate to use. > > Including the dependencies below as you suggest puts Hama in the place > of maintaining a transitive dependency like commons-cli in lock-step > hadoop-core, where the actual dependency comes from. > > > This is all just my $.02 worth, I'm open to arguments in favor of > <excludeTransitive/>, and there is often more than one way to skin a > cat with respect to dependency management. > > > ...joe > > > > > <dependency> > > <groupId>org.mortbay.jetty</groupId> > > <artifactId>jetty-annotations</artifactId> > > <version>6.1H.22</version> > > </dependency> > > <dependency> > > <groupId>org.slf4j</groupId> > > <artifactId>slf4j-log4j12</artifactId> > > <version>1.5.2</version> > > </dependency> > > <dependency> > > <groupId>commons-cli</groupId> > > <artifactId>commons-cli</artifactId> > > <version>1.2</version> > > </dependency> > > > > Thanks. > > > > On Thu, Jun 23, 2011 at 7:00 AM, Edward J. Yoon <[email protected]> > > wrote: > >> Oh... sorry, you're right. > >> > >> It looks like a bug. We have to fix this. > >> > >> [INFO] Total time: 4:36.098s > >> [INFO] Finished at: Thu Jun 23 06:58:28 KST 2011 > >> [INFO] Final Memory: 20M/51M > >> [INFO] > >> ------------------------------------------------------------------------ > >> edward@test ~/workspace/hama-trunk $ ls lib/ > >> commons-logging-1.0.4.jar hadoop-test-0.20.2.jar > >> jetty-util-6.1H.22.jar log4j-1.2.15.jar > >> hadoop-core-0.20.2.jar jetty-6.1H.22.jar junit-3.8.1.jar > >> zookeeper-3.3.1.jar > >> edward@tweetple ~/workspace/hama-trunk $ bin/hama zookeeper > >> -bash: bin/hama: Permission denied > >> edward@test ~/workspace/hama-trunk $ chmod 777 bin/** > >> edward@test ~/workspace/hama-trunk $ bin/hama zookeeper > >> Exception in thread "main" java.lang.NoClassDefFoundError: > >> org/apache/commons/cli/ParseException > >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59) > >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > >> at org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41) > >> Caused by: java.lang.ClassNotFoundException: > >> org.apache.commons.cli.ParseException > >> at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > >> at java.security.AccessController.doPrivileged(Native Method) > >> at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:303) > >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > >> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316) > >> ... 3 more > >> > >> > >> > >> On Thu, Jun 23, 2011 at 6:45 AM, Edward J. Yoon <[email protected]> > >> wrote: > >>> That's so weird. > >>> > >>> It works well on my test cluster. > >>> > >>> edward@slave:~/workspace/hama-trunk$ ls lib/ > >>> commons-logging-1.0.4.jar hadoop-test-0.20.2.jar > >>> jetty-util-6.1H.22.jar log4j-1.2.15.jar > >>> hadoop-core-0.20.2.jar jetty-6.1H.22.jar junit-3.8.1.jar > >>> zookeeper-3.3.1.jar > >>> edward@slave:~/workspace/hama-trunk$ bin/hama jar > >>> hama-examples-0.3.0-incubating-SNAPSHOT.jar pi > >>> 11/06/23 06:41:55 DEBUG bsp.BSPJobClient: BSPJobClient.submitJobDir: > >>> hdfs://slave.udanax.org:9000/tmp/hadoop-edward/bsp/system/submit_ebr5do > >>> 11/06/23 06:41:57 INFO bsp.BSPJobClient: Running job: > >>> job_201106230640_0001 > >>> 11/06/23 06:42:00 INFO bsp.BSPJobClient: Current supersteps number: 0 > >>> 11/06/23 06:42:03 INFO bsp.BSPJobClient: Current supersteps number: 1 > >>> 11/06/23 06:42:03 INFO bsp.BSPJobClient: The total number of supersteps: 1 > >>> Estimated value of PI is 3.1534 > >>> Job Finished in 7.886 seconds > >>> edward@slave:~/workspace/hama-trunk$ > >>> > >>> On Thu, Jun 23, 2011 at 4:21 AM, Joseph Boyd > >>> <[email protected]> wrote: > >>>> Replying to my own post ... > >>>> > >>>> On Wed, Jun 22, 2011 at 11:05 AM, Joseph Boyd > >>>> <[email protected]> wrote: > >>>>> On Thu, Jun 16, 2011 at 4:37 AM, Ashish Agarwal > >>>>> <[email protected]> wrote: > >>>>>> > >>>>>> When I try to run the hama daemons using the command - > >>>>>> bin/hama zookeeper > >>>>>> > >>>>>> I get this error - > >>>> > >>>> It looks like the problem is actually this line in pom.xml (line 205) : > >>>> <excludeTransitive>true</excludeTransitive> > >>>> > >>>> This prevents commons-cli (and other transitive dependencies) from > >>>> being copied to the lib dir like they should be. > >>>> > >>>> Removing line 205 fixes the NoClassDefFoundError for me. I can open a > >>>> JIRA for this after lunch, the command line is fairly broken with this > >>>> line in, especially after several dependencies were (correctly) > >>>> removed in HAMA-402. > >>>> > >>>> > >>>> ...joe > >>>> > >>>> > >>>>> I'm getting the same NoClassDefFoundError using the svn code. The > >>>>> commons-cli dependency was removed recently as part of HAMA-402, and > >>>>> re-adding it fixes the NoClassDefFoundError. > >>>>> > >>>>> I believe commons-cli should have been pulled as part of the > >>>>> hadoop-core dependency, and wasn't. (though my maven skills are > >>>>> rusty, and I'm no longer sure just how transitive dependencies are > >>>>> pulled) > >>>>> > >>>>> While a quick work-around is to re-add the dependency here, the proper > >>>>> fix is probably to fix the hadoop-core pom, and properly mark that it > >>>>> depends on commons-cli. > >>>>> > >>>>> > >>>>> ...joe > >>>>> > >>>>> > >>>>>> Exception in thread "main" java.lang.NoClassDefFoundError: > >>>>>> org/apache/commons/cli/ParseException > >>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59) > >>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > >>>>>> at org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41) > >>>>>> Caused by: java.lang.ClassNotFoundException: > >>>>>> org.apache.commons.cli.ParseException > >>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > >>>>>> at java.security.AccessController.doPrivileged(Native Method) > >>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > >>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > >>>>>> ... 3 more > >>>>>> > >>>>>> > >>>>>> I tried installing it a few times, but can't think of a solution to the > >>>>>> problem. Can you please help ? > >>>>>> > >>>>>> Thank you > >>>>>> > >>>>>> Sincerely, > >>>>>> Ashish > >>>>>> > >>>>>> > >>>>>> On Wed, Jun 15, 2011 at 3:18 AM, Edward J. Yoon > >>>>>> <[email protected]>wrote: > >>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> I would suggest following steps to debug: > >>>>>>> > >>>>>>> 1. Install the Hadoop 0.20.x on your laptop. > >>>>>>> 2. Run the Hama deamons with 3 terminals. > >>>>>>> > >>>>>>> % bin/hama zookeeper > >>>>>>> % bin/hama bspmaster > >>>>>>> % bin/hama groom > >>>>>>> > >>>>>>> 3. Use the logging API to log debug. > >>>>>>> For example, LOG.debug("like this"); > >>>>>>> > >>>>>>> On Wed, Jun 15, 2011 at 6:48 PM, Ashish Agarwal > >>>>>>> <[email protected]> > >>>>>>> wrote: > >>>>>>> > Hi, > >>>>>>> > > >>>>>>> > I was wondering if anyone can give me instructions on how to debug > >>>>>>> > code > >>>>>>> in > >>>>>>> > distributed or pseudo-distributed mode ? It would really help me. > >>>>>>> > > >>>>>>> > Thank you > >>>>>>> > > >>>>>>> > Sincerely, > >>>>>>> > Ashish > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> > On Sat, Jun 11, 2011 at 3:41 AM, Ashish Agarwal > >>>>>>> > <[email protected] > >>>>>>> >wrote: > >>>>>>> > > >>>>>>> >> > >>>>>>> >> Hi, > >>>>>>> >> > >>>>>>> >> I have written some code for my GSoC task (Hama-367) and am now > >>>>>>> >> trying > >>>>>>> to > >>>>>>> >> debug. > >>>>>>> >> > >>>>>>> >> I am trying to set up the environment into pseudo-distributed mode > >>>>>>> >> so I > >>>>>>> can > >>>>>>> >> debug. I think the value of bsp.master.address in hama-site.xml > >>>>>>> >> has to > >>>>>>> be > >>>>>>> >> local as this means LocalBSPRunner (as localhost would call > >>>>>>> >> 127.0.0.1 > >>>>>>> >> server), but where do I specify that in the code in eclipse ? > >>>>>>> >> > >>>>>>> >> My next question is - what is the difference between > >>>>>>> >> MiniBSPCluster and > >>>>>>> >> LocalBSPRunner and which file should I be using to be able to step > >>>>>>> through > >>>>>>> >> the code ? > >>>>>>> >> > >>>>>>> >> These are the processes currently running on my machine - > >>>>>>> >> > >>>>>>> >> 6112 NameNode > >>>>>>> >> 3319 ZooKeeperRunner > >>>>>>> >> 3018 > >>>>>>> >> 6449 SecondaryNameNode > >>>>>>> >> 5709 HMaster > >>>>>>> >> 5643 HQuorumPeer > >>>>>>> >> 6684 TaskTracker > >>>>>>> >> 6526 JobTracker > >>>>>>> >> 8074 Jps > >>>>>>> >> 6277 DataNode > >>>>>>> >> > >>>>>>> >> Thank you, > >>>>>>> >> Ashish > >>>>>>> >> > >>>>>>> > > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Best Regards, Edward J. Yoon > >>>>>>> @eddieyoon > >>>>>>> > >>>>>> > >>>>> > >>>> > >>> > >>> > >>> > >>> -- > >>> Best Regards, Edward J. Yoon > >>> @eddieyoon > >>> > >> > >> > >> > >> -- > >> Best Regards, Edward J. Yoon > >> @eddieyoon > >> > > > > > > > > -- > > Best Regards, Edward J. Yoon > > @eddieyoon > >
