On Mon, Feb 3, 2014 at 9:06 PM, Sebastian Schelter <[email protected]> wrote:
> If you just wanna play with Giraph, I suggest you start with the PageRank > example: > > https://giraph.apache.org/pagerank.html I have seen it. I think there is no much difference in SSSP and PageRank examples. By the way, I have not found LongDoubleNullTextInputFormat in my sources (I am using release-1.0.0-RC2-152-g585511f) which is mentioned there. The same issue with SimplePageRankComputation. [hduser@localhost giraph]$ $HADOOP_HOME/bin/hadoop jar $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimplePageRankComputation -vif org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip /user/hduser/input/test -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/output/amazon-shortestpaths -w 1 14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one. 14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one. Exception in thread "main" java.lang.IllegalArgumentException: checkClassTypes: vertex value types not assignable, computation - class org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class org.apache.hadoop.io.LongWritable at org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381) at org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228) at org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141) at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214) at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > > --sebastian > > > On 02/03/2014 05:49 PM, Alexander Frolov wrote: > >> I've tried .LongLongNullTextInputFormat and got this: >> >> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar >> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0- >> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar >> org.apache.giraph.GiraphRunner >> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip >> /user/hduser/input/test -vof >> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >> /user/hduser/output/amazon-shortestpaths -w 1 >> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge input format >> specified. Ensure your InputFormat does not require one. >> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge output format >> specified. Ensure your OutputFormat does not require one. >> Exception in thread "main" java.lang.IllegalArgumentException: >> checkClassTypes: vertex value types not assignable, computation - class >> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class >> org.apache.hadoop.io.LongWritable >> at >> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable( >> GiraphConfigurationValidator.java:381) >> at >> org.apache.giraph.job.GiraphConfigurationValidator. >> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator. >> java:228) >> at >> org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration( >> GiraphConfigurationValidator.java:141) >> at >> org.apache.giraph.utils.ConfigurationUtils.parseArgs( >> ConfigurationUtils.java:214) >> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke( >> NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke( >> DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >> >> >> >> >> On Mon, Feb 3, 2014 at 8:38 PM, Alexander Frolov >> <[email protected]>wrote: >> >> >>> >>> >>> On Mon, Feb 3, 2014 at 8:30 PM, Sebastian Schelter <[email protected]> >>> wrote >>> >>> What types does the vertex class that you use in your computation >>>> require? >>>> >>> >>> >>> I see. >>> >>> public class SimpleShortestPathsComputation extends BasicComputation< >>> LongWritable, DoubleWritable, FloatWritable, DoubleWritable> {..} >>> >>> So that is a mistake. >>> Thank you very much. >>> >>> >> Is there any means to import data from file like this (amazon0601.txt >>> from >>> SNAP data collection): >>> >>> 0 1 >>> 0 2 >>> 0 3 >>> 0 4 >>> 0 5 >>> 0 6 >>> 0 7 >>> 0 8 >>> 0 9 >>> 0 10 >>> ... >>> >>> and to perform basic tests from giraph-examples? Most of the tests are >>> using LongWritable for vertex id. >>> >>> >>> >>>> On 02/03/2014 05:28 PM, Alexander Frolov wrote: >>>> >>>> On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <[email protected]> >>>>> wrote: >>>>> >>>>> Hi Alexander, >>>>> >>>>>> >>>>>> what do you use as type for your vertex ids? It looks that you are >>>>>> trying >>>>>> to use longs, while IntIntNullTextInputFormat only provides ints, that >>>>>> could be the error. >>>>>> >>>>>> --sebastian >>>>>> >>>>>> >>>>>> I also thought so, but I could not understand why it regard it as >>>>>> long. >>>>>> >>>>> I just use -vif org.apache.giraph.io.formats. >>>>> IntIntNullTextInputFormat >>>>> -vip /user/hduser/input/test -vof >>>>> >>>>> and file consists only of two lines: >>>>> >>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat >>>>> /user/hduser/input/test >>>>> 0 1 >>>>> 1 0 >>>>> >>>>> I dont know any other means to specify type of vertex ids. >>>>> >>>>> >>>>> >>>>> On 02/03/2014 05:06 PM, Alexander Frolov wrote: >>>>>> >>>>>> Hello, >>>>>> >>>>>>> >>>>>>> I am trying to load data in from file in IntIntNullTextInputFormat. >>>>>>> But I >>>>>>> get is the following message: >>>>>>> >>>>>>> >>>>>>> >>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar >>>>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0- >>>>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar >>>>>>> org.apache.giraph.GiraphRunner >>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip >>>>>>> /user/hduser/input/test -vof >>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>> /user/hduser/output/amazon-shortestpaths -w 1 >>>>>>> >>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input format >>>>>>> specified. Ensure your InputFormat does not require one. >>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output >>>>>>> format >>>>>>> specified. Ensure your OutputFormat does not require one. >>>>>>> Exception in thread "main" java.lang.IllegalArgumentException: >>>>>>> checkClassTypes: vertex index types not assignable, computation - >>>>>>> class >>>>>>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class >>>>>>> org.apache.hadoop.io.IntWritable >>>>>>> at >>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable( >>>>>>> GiraphConfigurationValidator.java:381) >>>>>>> at >>>>>>> org.apache.giraph.job.GiraphConfigurationValidator. >>>>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator. >>>>>>> java:226) >>>>>>> at >>>>>>> org.apache.giraph.job.GiraphConfigurationValidator. >>>>>>> validateConfiguration( >>>>>>> GiraphConfigurationValidator.java:141) >>>>>>> at >>>>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs( >>>>>>> ConfigurationUtils.java:214) >>>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74) >>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >>>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>>> at >>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke( >>>>>>> NativeMethodAccessorImpl.java:57) >>>>>>> at >>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke( >>>>>>> DelegatingMethodAccessorImpl.java:43) >>>>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> The input file is here (just for testing): >>>>>>> >>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat >>>>>>> /user/hduser/input/test >>>>>>> 0 1 >>>>>>> 1 0 >>>>>>> >>>>>>> Could you please help with this issue? >>>>>>> >>>>>>> Best, >>>>>>> Alex >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
