Hi Claudio actually i did increased limit on my laptop to 1024 (pseudo mode) it took 361 supersteps for 15 nodes and 18 edges (depth 10+), we will soon be testing this on cluster of 5 or 6 hope everything goes well. Is there a way i can disable this check..? i remember someone posting this earlier..!! but not sure how..;-(
@Jan, yes, actually even i want to see results,..:-) PS thanks for all help.. On Thu, Feb 7, 2013 at 9:47 PM, Jan van der Lugt <[email protected]> wrote: > I never actually ran bc_random, so I don't know how many supersteps it > needs for different input sizes. If you could try this out, it would be > worthwhile to know. Especially the relation between input size / number of > supersteps. It's a limitation of the Pregel paradigm that you need quite a > few superstep to do BFS, not a lot we can do about this... > > > On Thu, Feb 7, 2013 at 3:47 PM, Claudio Martella < > [email protected]> wrote: > >> Hi, >> >> you have to increase the counters limit in your cluster. You can increase >> the limit by setting the property mapreduce.job.counters.limit in your >> mapred-site.xml accordingly. >> >> >> On Thu, Feb 7, 2013 at 4:34 PM, pradeep kumar <[email protected]>wrote: >> >>> Hi Jan, >>> >>> well actually i just corrected my mistake.. i was running it wrong way, >>> thanks.. >>> just 1 more question on map-reduce counters >>> >>> giraph by default limits the superstep counters to 120 >>> but in bc_random master superstep itself exceeds this, so i had to >>> increase the limit..this was just for 20 nodes sample i/p i was trying >>> >>> my real data will be close to million nodes, so how can i manage for >>> such large input.. >>> >>> >>> >>> >>> On Thu, Feb 7, 2013 at 4:02 PM, pradeep kumar <[email protected]>wrote: >>> >>>> Hi lugt, >>>> >>>> Thanks for all your help.. program compiled successfully into jar.. >>>> but actually i am not able to run programs.. >>>> i tried following cmd to run >>>> >>>> 1) using GiraphRunner >>>> hadoop jar >>>> giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar >>>> org.apache.giraph.GiraphRunner org.apache.giraph.examples.bc_randomVertex >>>> -vif org.apache.giraph.examples.bc_randomVertexInputFormat -vip inputbet >>>> -of org.apache.giraph.examples.bc_randomVertexOutputFormat -op outputbet >>>> -w 1 >>>> >>>> it does maps every thing successfully but then gives following error in >>>> log >>>> >>>> java.lang.IllegalStateException: run: Caught an unrecoverable exception >>>> waitFor: ExecutionException occurred while waiting for >>>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@40fd6cd8 >>>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102) >>>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647) >>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) >>>> at org.apache.hadoop.mapred.Child$4.run(Child.java:266) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at javax.security.auth.Subject.doAs(Subject.java:415) >>>> >>>> followed by some other errors >>>> >>>> 2) bc_random >>>> >>>> i actually couldnt understand all arg types here >>>> >>>> used : >>>> >>>> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar >>>> org.apache.giraph.examples.bc_random >>>> org.apache.giraph.examples.bc_randomVertex --GMInputFormat ADJ >>>> org.apache.giraph.examples.bc_randomVertexInputFormat -i inputbet >>>> --GMOutputFormat ADJ >>>> org.apache.giraph.examples.bc_randomVertexOutputFormat -o outputbet -w 1 >>>> >>>> >>>> >>>> >>>> but gives error for k procedure arg (not sure what to give as value here, >>>> even about about ip format i mentioned above ) >>>> >>>> >>>> I have uploaded sample input in hdfs in following format >>>> >>>> 10 4500 9 800 11 1000 12 1000 >>>> 11 5500 12 1100 11 800 >>>> >>>> nid val <eid val> >>>> >>>> did i made a mistake anywhere..? >>>> >>>> any suggestion will be a great help..!! >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Feb 6, 2013 at 5:03 PM, Jan van der Lugt <[email protected]>wrote: >>>> >>>>> Hi Pradeep, >>>>> >>>>> You can check out the Green-Marl compiler from my Github ( >>>>> https://github.com/janlugt/Green-Marl), that one should be compatible >>>>> with the latest version of Giraph. Please let me know if anything doesn't >>>>> work for you. The changes will probably also be accepted upstream later >>>>> this day. >>>>> >>>>> - Jan >>>>> >>>>> >>>>> On Wed, Feb 6, 2013 at 9:48 AM, Jan van der Lugt <[email protected]>wrote: >>>>> >>>>>> It should work with version 0.2, but let me check. Since the API is >>>>>> changing all the time at the moment, it's very different to track all >>>>>> those >>>>>> changes. Probably they are numerous instances of the same error. >>>>>> >>>>>> >>>>>> On Wed, Feb 6, 2013 at 9:26 AM, pradeep kumar >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> Hi Lugt, >>>>>>> >>>>>>> Thanks for suggestion, i just tried with Green-Marl's bc_random, >>>>>>> But because of so many changes in Giraph codebase, imports and many >>>>>>> other functions in bc_random >>>>>>> generate lots of errors almost 100+ errors.. I tried with Giraph 1.0 >>>>>>> and current version of 2.0, >>>>>>> most of errors i have resolved but still there are many errors.. >>>>>>> Any suggestion on this..? >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Pradeep Kumar >>>> >>> >>> >>> >>> -- >>> Pradeep Kumar >>> >> >> >> >> -- >> Claudio Martella >> [email protected] >> > > -- Pradeep Kumar
