but you were still using an additional jar added through -libjars, right?
On Mon, Aug 26, 2013 at 8:43 AM, Vivek Sembium <[email protected]>wrote: > @Claudio Martella Your solution didnt work either. I basically tried > copying the pageRankBenchmark to my own package, renamed the package. It > compiles fine with giraph. But I couldnt run it even if I add those files > to giraph jar before deployment. Help? > > > On Sun, Aug 25, 2013 at 6:33 PM, Claudio Martella < > [email protected]> wrote: > >> you have this problem when you use two jars (one with giraph and one with >> your classes) instead of a single fat-jar, correct? I tracked the same >> problem a few weeks ago, basically zookeeper is run passing the wrong jar. >> >> >> On Sat, Aug 24, 2013 at 4:51 PM, Vivek Sembium >> <[email protected]>wrote: >> >>> Thank you for your suggestion. It worked. Its not giving class not found >>> exception. But its giving me a new error >>> Its stopping at map 0% and reduce 0%. Upon inspection I found that its >>> unable to connect to zookeeper service. >>> >>> java.lang.IllegalStateException: run: Caught an unrecoverable exception >>> onlineZooKeeperServers: Failed to connect in 10 tries! >>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101) >>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) >>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) >>> at org.apache.hadoop.mapred.Child$4.run(Child.java:255) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:416) >>> at >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) >>> at org.apache.hadoop.mapred.Child.main(Child.java:249) >>> Caused by: java.lang.IllegalStateException: onlineZooKeeperServers: >>> Failed to connect in 10 tries! >>> at >>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:727) >>> at >>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:371) >>> at >>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:204) >>> at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59) >>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89) >>> ... 7 more >>> >>> Immediately I ran page rank benchmark and it executed successfully both >>> from giraph in lib directory and also from giraphs own directory. >>> >>> Can you give me a very simple java program(finding maximum in a graph or >>> simple page rank program) in giraph along with its jar file and input files >>> which I can place in my lib directory of hadoop and test if its working. >>> And also the command to execute it. This should be added in the >>> documentation as new comers can quickly setup giraph and concentrate on >>> their project. >>> >>> >>> On Sat, Aug 24, 2013 at 7:12 PM, Ahmet Emre Aladağ < >>> [email protected]> wrote: >>> >>>> It isn't asking for edge input. It says make sure you don't need it. A >>>> warning for the case you may have forgotten to give edge input when you >>>> really needed. >>>> >>>> The cause of your error is what I'm wondering nowadays. I'm having a >>>> similar problem. Currently I'm using a workaround: put all the jars >>>> (giraph-core and my module giraph-nutch) in the lib folder of hadoop. Then >>>> it works. But there should be a clean way of doing this. >>>> >>>> I should be able to say hadoop jar fat.jar ... >>>> >>>> Any help appreciated. >>>> >>>> >>>> >>>> >>>> >>>> ------------------------------ >>>> *Kimden: *"Vivek Sembium" <[email protected]> >>>> *Kime: *[email protected] >>>> *Gönderilenler: *24 Ağustos Cumartesi 2013 11:51:49 >>>> *Konu: *Re: Help needed for Running my own java programs in Giraph >>>> >>>> >>>> I tried with and without exporting hadoop classpath. I get the same >>>> error. >>>> >>>> Here's the command that I tried >>>> hadoop jar >>>> /mnt/a1/sda4/hadoop/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar >>>> org.apache.giraph.GiraphRunner -libjars >>>> /mnt/a99/d0/vivek/workspace/Giraph/bin/SimplePageRankComputation.jar >>>> practice.SimplePageRankComputation\$SimplePageRankMasterCompute -vif >>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>> -vip /user/vivek/tiny_graph.txt -w 3 >>>> >>>> >>>> 13/08/24 14:21:00 INFO utils.ConfigurationUtils: No edge input format >>>> specified. Ensure your InputFormat does not require one. >>>> 13/08/24 14:21:00 INFO utils.ConfigurationUtils: No output format >>>> specified. Ensure your OutputFormat does not require one. >>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>> org/apache/giraph/master/DefaultMasterCompute >>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:634) >>>> at >>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:277) >>>> at java.net.URLClassLoader.access$000(URLClassLoader.java:73) >>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:212) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:321) >>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:314) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:266) >>>> at java.lang.Class.forName0(Native Method) >>>> at java.lang.Class.forName(Class.java:186) >>>> at >>>> org.apache.giraph.utils.ConfigurationUtils.handleComputationClass(ConfigurationUtils.java:425) >>>> at >>>> org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:408) >>>> at >>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:200) >>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74) >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:616) >>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >>>> Caused by: java.lang.ClassNotFoundException: >>>> org.apache.giraph.master.DefaultMasterCompute >>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:217) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:321) >>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:266) >>>> ... 26 more >>>> >>>> >>>> I gave the graph input as JSON file. Why does it ask for edge input >>>> file again? >>>> >>>> >>>> Why isnt this working? >>>> >>>> >>>> On Sat, Aug 24, 2013 at 12:35 AM, Kyle Orlando < >>>> [email protected]> wrote: >>>> >>>>> Hi Vivek, >>>>> >>>>> Sorry for the hasty response, I assumed the quick start guide covered >>>>> that (I guess it doesn't). What you need to do first is put all of your >>>>> compiled code into a jar file. Then, you'll be utilizing the hadoop >>>>> option >>>>> -libjars. You also need to make sure that you set the hadoop classpath to >>>>> include the giraph jar you are using as well as your own jar. I actually >>>>> asked a similar question not too long ago; prior to this, I was just >>>>> combining everything into one jar. >>>>> >>>>> >>>>> Your code should pretty much follow this format, which is copied and >>>>> paster from a previous response by Claudio: >>>>> >>>>> export HADOOP_CLASSPATH="giraph.jar:your.jar" >>>>> hadoop jar giraph.jar org.apache.giraph.GiraphRunner [ -D option ]* >>>>> -libjars your.jar your.package.ComputationClass [ GiraphRunner option >>>>> e.g. -vip -vif etc. ]* >>>>> >>>>> >>>>> On Fri, Aug 23, 2013 at 2:51 PM, Vivek Sembium < >>>>> [email protected]> wrote: >>>>> >>>>>> Thank You for your response. I could run all the example programs. I >>>>>> want to place my development code in a different directory other than the >>>>>> giraph directory. I copied the SimplePageRankComputation.java to a >>>>>> different directory, compiled it(with a different package name). But I >>>>>> couldnt replace the >>>>>> "org.apache.giraph.examples.SimpleShortestPathsComputation" >>>>>> with my class file. It gives me a ClassNotFoundException. Please Guide me >>>>>> so that I can run any class file with all required methods (GIven the >>>>>> location of that file). >>>>>> >>>>>> >>>>>> On Sat, Aug 24, 2013 at 12:07 AM, Kyle Orlando < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> A "Quick Start" guide was recently added to the Giraph website. Now >>>>>>> learning Giraph is easier than ever! Here: >>>>>>> http://giraph.apache.org/quick_start.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Aug 23, 2013 at 2:34 PM, Vivek Sembium < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> I am a new user in giraph. I could run all example programs in >>>>>>>> Giraph. Can someone please guide me on how to run my own java program >>>>>>>> in >>>>>>>> giraph (Assuming I have my input file in jason format in HDFS). >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Kyle Orlando >>>>>>> Computer Engineering Major >>>>>>> University of Maryland >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Kyle Orlando >>>>> Computer Engineering Major >>>>> University of Maryland >>>>> >>>> >>>> >>>> >>> >> >> >> -- >> Claudio Martella >> [email protected] >> > > -- Claudio Martella [email protected]
