Thanks a lot. One last question: where do I set options like USE_SUPERSTEP_COUNTERS?
Christian 2013/9/9 André Kelpe <[email protected]> > On older versions of hadoop, you cannot set the counters to a higher > value. That was only introduced later. I had this issue on CDH3 (~1.5 > years ago) and my solution was to disable all counters for the giraph > job, to make it work. If you use a more modern version of hadoop, it > should be possible to increase the limit though. > > - André > > 2013/9/9 Avery Ching <[email protected]>: > > If you are running out of counters, you can turn off the superstep > counters > > > > /** Use superstep counters? (boolean) */ > > BooleanConfOption USE_SUPERSTEP_COUNTERS = > > new BooleanConfOption("giraph.useSuperstepCounters", true, > > "Use superstep counters? (boolean)"); > > > > > > On 9/9/13 6:43 AM, Claudio Martella wrote: > > > > No, I used a different counters limit on that hadoop version. Setting > > mapreduce.job.counters.limit to a higher number and restarting JT and TT > > worked for me. Maybe 64000 might be too high? Try setting it to 512. Does > > not look like the case, but who knows. > > > > > > On Mon, Sep 9, 2013 at 2:57 PM, Christian Krause <[email protected]> wrote: > >> > >> Sorry, it still doesn't work (I ran into a different problem before I > >> reached the limit). > >> > >> I am using Hadoop 0.20.203.0. Is the limit of 120 counters maybe > >> hardcoded? > >> > >> Cheers > >> Christian > >> > >> Am 09.09.2013 08:29 schrieb "Christian Krause" <[email protected]>: > >> > >>> I changed the property name to mapred.job.counters.limit and restarted > it > >>> again. Now it works. > >>> > >>> Thanks, > >>> Christian > >>> > >>> > >>> 2013/9/7 Claudio Martella <[email protected]> > >>>> > >>>> did you restart TT and JT? > >>>> > >>>> > >>>> On Sat, Sep 7, 2013 at 7:09 AM, Christian Krause <[email protected]> > wrote: > >>>>> > >>>>> Hi, > >>>>> I've increased the counter limit in mapred-site.xml, but I still get > >>>>> the error: Exceeded counter limits - Counters=121 Limit=120. Groups=6 > >>>>> Limit=50. > >>>>> > >>>>> This is my config: > >>>>> > >>>>> cat conf/mapred-site.xml > >>>>> <?xml version="1.0"?> > >>>>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > >>>>> > >>>>> <!-- Put site-specific property overrides in this file. --> > >>>>> > >>>>> <configuration> > >>>>> ... > >>>>> <property> > >>>>> <name>mapreduce.job.counters.limit</name> > >>>>> <value>64000</value> > >>>>> </property> > >>>>> <property> > >>>>> <name>mapred.task.timeout</name> > >>>>> <value>2400000</value> > >>>>> </property> > >>>>> ... > >>>>> </configuration> > >>>>> > >>>>> Any ideas? > >>>>> > >>>>> Cheers, > >>>>> Christian > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Claudio Martella > >>>> [email protected] > >>> > >>> > > > > > > > > -- > > Claudio Martella > > [email protected] > > > > >
