I never actually ran bc_random, so I don't know how many supersteps it needs for different input sizes. If you could try this out, it would be worthwhile to know. Especially the relation between input size / number of supersteps. It's a limitation of the Pregel paradigm that you need quite a few superstep to do BFS, not a lot we can do about this...
On Thu, Feb 7, 2013 at 3:47 PM, Claudio Martella <[email protected] > wrote: > Hi, > > you have to increase the counters limit in your cluster. You can increase > the limit by setting the property mapreduce.job.counters.limit in your > mapred-site.xml accordingly. > > > On Thu, Feb 7, 2013 at 4:34 PM, pradeep kumar <[email protected]>wrote: > >> Hi Jan, >> >> well actually i just corrected my mistake.. i was running it wrong way, >> thanks.. >> just 1 more question on map-reduce counters >> >> giraph by default limits the superstep counters to 120 >> but in bc_random master superstep itself exceeds this, so i had to >> increase the limit..this was just for 20 nodes sample i/p i was trying >> >> my real data will be close to million nodes, so how can i manage for >> such large input.. >> >> >> >> >> On Thu, Feb 7, 2013 at 4:02 PM, pradeep kumar <[email protected]>wrote: >> >>> Hi lugt, >>> >>> Thanks for all your help.. program compiled successfully into jar.. >>> but actually i am not able to run programs.. >>> i tried following cmd to run >>> >>> 1) using GiraphRunner >>> hadoop jar >>> giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar >>> org.apache.giraph.GiraphRunner org.apache.giraph.examples.bc_randomVertex >>> -vif org.apache.giraph.examples.bc_randomVertexInputFormat -vip inputbet >>> -of org.apache.giraph.examples.bc_randomVertexOutputFormat -op outputbet >>> -w 1 >>> >>> it does maps every thing successfully but then gives following error in >>> log >>> >>> java.lang.IllegalStateException: run: Caught an unrecoverable exception >>> waitFor: ExecutionException occurred while waiting for >>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@40fd6cd8 >>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102) >>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647) >>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) >>> at org.apache.hadoop.mapred.Child$4.run(Child.java:266) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:415) >>> >>> followed by some other errors >>> >>> 2) bc_random >>> >>> i actually couldnt understand all arg types here >>> >>> used : >>> >>> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar >>> org.apache.giraph.examples.bc_random >>> org.apache.giraph.examples.bc_randomVertex --GMInputFormat ADJ >>> org.apache.giraph.examples.bc_randomVertexInputFormat -i inputbet >>> --GMOutputFormat ADJ >>> org.apache.giraph.examples.bc_randomVertexOutputFormat -o outputbet -w 1 >>> >>> >>> >>> but gives error for k procedure arg (not sure what to give as value here, >>> even about about ip format i mentioned above ) >>> >>> >>> I have uploaded sample input in hdfs in following format >>> >>> 10 4500 9 800 11 1000 12 1000 >>> 11 5500 12 1100 11 800 >>> >>> nid val <eid val> >>> >>> did i made a mistake anywhere..? >>> >>> any suggestion will be a great help..!! >>> >>> >>> >>> >>> >>> On Wed, Feb 6, 2013 at 5:03 PM, Jan van der Lugt <[email protected]>wrote: >>> >>>> Hi Pradeep, >>>> >>>> You can check out the Green-Marl compiler from my Github ( >>>> https://github.com/janlugt/Green-Marl), that one should be compatible >>>> with the latest version of Giraph. Please let me know if anything doesn't >>>> work for you. The changes will probably also be accepted upstream later >>>> this day. >>>> >>>> - Jan >>>> >>>> >>>> On Wed, Feb 6, 2013 at 9:48 AM, Jan van der Lugt <[email protected]>wrote: >>>> >>>>> It should work with version 0.2, but let me check. Since the API is >>>>> changing all the time at the moment, it's very different to track all >>>>> those >>>>> changes. Probably they are numerous instances of the same error. >>>>> >>>>> >>>>> On Wed, Feb 6, 2013 at 9:26 AM, pradeep kumar >>>>> <[email protected]>wrote: >>>>> >>>>>> Hi Lugt, >>>>>> >>>>>> Thanks for suggestion, i just tried with Green-Marl's bc_random, >>>>>> But because of so many changes in Giraph codebase, imports and many >>>>>> other functions in bc_random >>>>>> generate lots of errors almost 100+ errors.. I tried with Giraph 1.0 >>>>>> and current version of 2.0, >>>>>> most of errors i have resolved but still there are many errors.. >>>>>> Any suggestion on this..? >>>>>> >>>>> >>>>> >>>> >>> >>> >>> -- >>> Pradeep Kumar >>> >> >> >> >> -- >> Pradeep Kumar >> > > > > -- > Claudio Martella > [email protected] >
