Hi Laxmi, I think you should tune your hbase. This informations is not enough for understanding your problem. you can add below lines in $HBASE_HOME/conf/hbase-env.sh for debuging.
export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:/usr/local/hbase/logs/gc-hbase.log" In our cluster, I use Concurrent-Mark-Sweep GC(CMS). Just for training you add your hbase-env.sh below lines: export HBASE_OPTS="$HBASE_OPTS -XX:CMSInitiatingOccupancyFraction=60" I think these configuration will be solved your problem. But if you want to learn how to tune my hbase below documents will be good start point. http://www.packtpub.com/article/hbase-basic-performance-tuning http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html I hope you solve your problems :) Talat ----- Orijinal Mesaj ----- Kimden: A Laxmi <[email protected]> Kime: [email protected] Gönderilenler: Tue, 22 Oct 2013 02:47:20 +0300 (EEST) Konu: [Nutch 2.2.1] Error java.lang.OutOfMemoryError: GC overhead limit exceeded Hi, I saw this strange out of memory error today that said GC overhead limit exceeded -* Error java.lang.OutOfMemoryError: GC overhead limit exceeded *in parsing stage. I have seen OOM errors; heap space before but never that said "GC overhead limit exceeded" Below is the log. Can anyone please comment? 2013-10-21 08:59:33,879 INFO parse.ParserJob - Parsing http://www.###.##.###/pub/###2011.pdf 2013-10-21 09:02:01,697 ERROR zookeeper.ClientCnxn - from pool-1-thread-1-SendThread(isscvm.isscdomain:2181) *java.lang.OutOfMemoryError: GC overhead limit exceeded* at java.util.Arrays.copyOf(Arrays.java:2367) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130) at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415) at java.lang.StringBuilder.append(StringBuilder.java:132) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1157) 2013-10-21 09:05:55,650 WARN parse.ParseUtil - Error parsing http://www.fs.fed.us/pnw/olympia/silv/publications/opt/622_HarringtonDevine2011.pdf java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:202) at org.apache.nutch.parse.ParseUtil.runParser(ParseUtil.java:148) at org.apache.nutch.parse.ParseUtil.parse(ParseUtil.java:129) at org.apache.nutch.parse.ParseUtil.process(ParseUtil.java:176) at org.apache.nutch.parse.ParserJob$ParserMapper.map(ParserJob.java:131) at org.apache.nutch.parse.ParserJob$ParserMapper.map(ParserJob.java:78) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:223) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 2013-10-21 09:05:55,651 WARN parse.ParseUtil - Unable to successfully parse content http://www.fs.fed.us/pnw/olympia/silv/publications/opt/622_HarringtonDevine2011.pdfof type application/pdf 2013-10-21 09:05:44,429 ERROR zookeeper.ClientCnxn - from pool-1-thread-1-SendThread(centosvm.centosdomain:2181) java.lang.OutOfMemoryError: GC overhead limit exceeded 2013-10-21 09:05:38,851 ERROR zookeeper.ClientCnxn - from pool-1-thread-1-SendThread(centosvm.centosdomain:2181) java.lang.OutOfMemoryError: GC overhead limit exceeded 2013-10-21 09:05:27,363 ERROR zookeeper.ClientCnxn - Error while calling watcher java.lang.OutOfMemoryError: GC overhead limit exceeded 2013-10-21 09:05:23,990 ERROR zookeeper.ClientCnxn - from pool-1-thread-1-SendThread(centosvm.centosdomain:2181) java.lang.OutOfMemoryError: GC overhead limit exceeded 2013-10-21 09:05:14,381 ERROR zookeeper.ClientCnxn - from pool-1-thread-1-SendThread(centosvm.centosdomain:2181) java.lang.OutOfMemoryError: GC overhead limit exceeded 2013-10-21 09:05:09,330 ERROR zookeeper.ClientCnxn - Caught unexpected throwable java.lang.OutOfMemoryError: GC overhead limit exceeded
