Hi, I saw this strange out of memory error today that said GC overhead limit exceeded -* Error java.lang.OutOfMemoryError: GC overhead limit exceeded *in parsing stage. I have seen OOM errors; heap space before but never that said "GC overhead limit exceeded"
Below is the log. Can anyone please comment? 2013-10-21 08:59:33,879 INFO parse.ParserJob - Parsing http://www.###.##.###/pub/###2011.pdf 2013-10-21 09:02:01,697 ERROR zookeeper.ClientCnxn - from pool-1-thread-1-SendThread(isscvm.isscdomain:2181) *java.lang.OutOfMemoryError: GC overhead limit exceeded* at java.util.Arrays.copyOf(Arrays.java:2367) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130) at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415) at java.lang.StringBuilder.append(StringBuilder.java:132) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1157) 2013-10-21 09:05:55,650 WARN parse.ParseUtil - Error parsing http://www.fs.fed.us/pnw/olympia/silv/publications/opt/622_HarringtonDevine2011.pdf java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:202) at org.apache.nutch.parse.ParseUtil.runParser(ParseUtil.java:148) at org.apache.nutch.parse.ParseUtil.parse(ParseUtil.java:129) at org.apache.nutch.parse.ParseUtil.process(ParseUtil.java:176) at org.apache.nutch.parse.ParserJob$ParserMapper.map(ParserJob.java:131) at org.apache.nutch.parse.ParserJob$ParserMapper.map(ParserJob.java:78) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:223) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 2013-10-21 09:05:55,651 WARN parse.ParseUtil - Unable to successfully parse content http://www.fs.fed.us/pnw/olympia/silv/publications/opt/622_HarringtonDevine2011.pdfof type application/pdf 2013-10-21 09:05:44,429 ERROR zookeeper.ClientCnxn - from pool-1-thread-1-SendThread(centosvm.centosdomain:2181) java.lang.OutOfMemoryError: GC overhead limit exceeded 2013-10-21 09:05:38,851 ERROR zookeeper.ClientCnxn - from pool-1-thread-1-SendThread(centosvm.centosdomain:2181) java.lang.OutOfMemoryError: GC overhead limit exceeded 2013-10-21 09:05:27,363 ERROR zookeeper.ClientCnxn - Error while calling watcher java.lang.OutOfMemoryError: GC overhead limit exceeded 2013-10-21 09:05:23,990 ERROR zookeeper.ClientCnxn - from pool-1-thread-1-SendThread(centosvm.centosdomain:2181) java.lang.OutOfMemoryError: GC overhead limit exceeded 2013-10-21 09:05:14,381 ERROR zookeeper.ClientCnxn - from pool-1-thread-1-SendThread(centosvm.centosdomain:2181) java.lang.OutOfMemoryError: GC overhead limit exceeded 2013-10-21 09:05:09,330 ERROR zookeeper.ClientCnxn - Caught unexpected throwable java.lang.OutOfMemoryError: GC overhead limit exceeded
