Is it possible that your map class is an inner class and not static? On Tue, Jun 16, 2009 at 10:51 AM, akhil1988 <akhilan...@gmail.com> wrote:
> > Hi All, > > I am running my mapred program in local mode by setting > mapred.jobtracker.local to local mode so that I can debug my code. > The mapred program is a direct porting of my original sequential code. > There > is no reduce phase. > Basically, I have just put my program in the map class. > > My program takes around 1-2 min. in instantiating the data objects which > are > present in the constructor of Map class(it loads some data model files, > therefore it takes some time). After the instantiation part in the > constrcutor of Map class the map function is supposed to process the input > split. > > The problem is that the data objects do not get instantiated completely and > in between(whlie it is still in constructor) the program stops giving the > exceptions pasted at bottom. > The program runs fine without mapreduce and does not require more than 2GB > memory, but in mapreduce even after doing export HADOOP_HEAPSIZE=2500(I am > working on machines with 16GB RAM), the program fails. I have also set > HADOOP_OPTS="-server -XX:-UseGCOverheadLimit" as sometimes I was getting GC > Overhead Limit Exceeded exceptions also. > > Somebody, please help me with this problem: I have trying to debug it for > the last 3 days, but unsuccessful. Thanks! > > java.lang.OutOfMemoryError: Java heap space > at > sun.misc.FloatingDecimal.toJavaFormatString(FloatingDecimal.java:889) > at java.lang.Double.toString(Double.java:179) > at java.text.DigitList.set(DigitList.java:272) > at java.text.DecimalFormat.format(DecimalFormat.java:584) > at java.text.DecimalFormat.format(DecimalFormat.java:507) > at java.text.NumberFormat.format(NumberFormat.java:269) > at > org.apache.hadoop.util.StringUtils.formatPercent(StringUtils.java:110) > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1147) > at LbjTagger.NerTagger.main(NerTagger.java:109) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:165) > at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68) > > 09/06/16 12:34:41 WARN mapred.LocalJobRunner: job_local_0001 > java.lang.RuntimeException: java.lang.reflect.InvocationTargetException > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:81) > at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58) > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > at > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:79) > ... 5 more > Caused by: java.lang.ThreadDeath > at java.lang.Thread.stop(Thread.java:715) > at > org.apache.hadoop.mapred.LocalJobRunner.killJob(LocalJobRunner.java:310) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.killJob(JobClient.java:315) > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1224) > at LbjTagger.NerTagger.main(NerTagger.java:109) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:165) > at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68) > > -- > View this message in context: > http://www.nabble.com/Nor-%22OOM-Java-Heap-Space%22-neither-%22GC-OverHead-Limit-Exeeceded%22-tp24059508p24059508.html > Sent from the Hadoop core-user mailing list archive at Nabble.com. > > -- Pro Hadoop, a book to guide you from beginner to hadoop mastery, http://www.amazon.com/dp/1430219424?tag=jewlerymall www.prohadoopbook.com a community for Hadoop Professionals
