Debarshi Basak
Tata Consultancy Services
Mailto: debarshi.ba...@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Outsourcing
____________________________________________
-----alo alt
To: user@hive.apache.org
From: alo alt <wget.n...@googlemail.com>
Date: 05/23/2012 02:51PM
Subject: Re:Use the memory management options, as described in the link above. You was gotten OOM - out of memory - and that could depend on a misconfiguration. Did you try playing with mapred.child.ulimit and with java.opts? -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF On May 23, 2012, at 11:12 AM, Debarshi Basak wrote: > But what i am doing is i am creating index then setting the path of the index and running a select <columns> from table_name where <condition> > How can i resolve this issue? > > > Debarshi Basak > Tata Consultancy Services > Mailto: debarshi.ba...@tcs.com > Website: http://www.tcs.com > ____________________________________________ > Experience certainty. IT Services > Business Solutions > Outsourcing > ____________________________________________ > > -----alo alt wrote: ----- > To: user@hive.apache.org > From: alo alt <wget.n...@googlemail.com> > Date: 05/23/2012 02:08PM > Subject: Re: > > Hi, > > > http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html#Memory+management > > > This message means that for some reason the garbage collector is taking an excessive amount of time (by default 98% of all CPU time of the process) and recovers very little memory in each run (by default 2% of the heap). > > -- > Alexander Alten-Lorenz > > http://mapredit.blogspot.com > > German Hadoop LinkedIn Group: > http://goo.gl/N8pCF > > > On May 23, 2012, at 10:13 AM, Debarshi Basak wrote: > > > When i am trying to run a query with index i am getting this exception.My hive version is 0.7.1 > > > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > at java.nio.ByteBuffer.wrap(ByteBuffer.java:369) > > at org.apache.hadoop.io.Text.decode(Text.java:327) > > at org.apache.hadoop.io.Text.toString(Text.java:254) > > at org.apache.hadoop.hive.ql.index.compact.HiveCompactIndexResult.add(HiveCompactIndexResult.java:118) > > at org.apache.hadoop.hive.ql.index.compact.HiveCompactIndexResult.<init>(HiveCompactIndexResult.java:107) > > at org.apache.hadoop.hive.ql.index.compact.HiveCompactIndexInputFormat.getSplits(HiveCompactIndexInputFormat.java:89) > > at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:971) > > at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:963) > > at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170) > > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880) > > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:415) > > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) > > at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833) > > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807) > > at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:671) > > at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123) > > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:131) > > at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) > > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063) > > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900) > > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748) > > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:209) > > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:286) > > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:516) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:601) > > at org.apache.hadoop.util.RunJar.main(RunJar.java:197) > > > > > > > > Debarshi Basak > > Tata Consultancy Services > > Mailto: debarshi.ba...@tcs.com > > Website: > http://www.tcs.com > > > ____________________________________________ > > Experience certainty. IT Services > > Business Solutions > > Outsourcing > > ____________________________________________ > > =====-----=====-----===== > > Notice: The information contained in this e-mail > > message and/or attachments to it may contain > > confidential or privileged information. If you are > > not the intended recipient, any dissemination, use, > > review, distribution, printing or copying of the > > information contained in this e-mail message > > and/or attachments to it are strictly prohibited. If > > you have received this communication in error, > > please notify us by reply e-mail or telephone and > > immediately and permanently delete the message > > and any attachments. Thank you > > > > > >
