Hi, I think it depends a lot on the data volume you are trying to process. Does it work with a smaller table?
Missatge de Bitfox <bit...@bitfox.top> del dia dt., 29 de març 2022 a les 14:39: > 0: jdbc:hive2://localhost:10000/default> set hive.tez.container.size=1024; > > No rows affected (0.027 seconds) > > > 0: jdbc:hive2://localhost:10000/default> set hive.execution.engine; > > +---------------------------+ > > | set | > > +---------------------------+ > > | hive.execution.engine=mr | > > +---------------------------+ > > 1 row selected (0.048 seconds) > > > 0: jdbc:hive2://localhost:10000/default> set mapreduce.map.memory.mb=1024; > > No rows affected (0.032 seconds) > > 0: jdbc:hive2://localhost:10000/default> set > mapreduce.map.java.opts=-Xmx1024m; > > No rows affected (0.01 seconds) > > 0: jdbc:hive2://localhost:10000/default> set > mapreduce.reduce.memory.mb=1024; > > No rows affected (0.014 seconds) > > 0: jdbc:hive2://localhost:10000/default> set > mapreduce.reduce.java.opts=-Xmx1024m; > > No rows affected (0.015 seconds) > > > 0: jdbc:hive2://localhost:10000/default> select job,count(*) as dd from > ppl group by job limit 10; > > Error: Error while processing statement: FAILED: Execution Error, return > code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask > (state=08S01,code=2) > > > > > Sorry my test VM has 2gb Ram only. So I set all the above memory size to > 1GB. > > But it still gets the same error. > > > > please help. thanks. > > > > On Tue, Mar 29, 2022 at 8:32 PM Pau Tallada <tall...@pic.es> wrote: > >> I assume you have to increase container size (if using tez/yarn) >> >> Missatge de Bitfox <bit...@bitfox.top> del dia dt., 29 de març 2022 a >> les 14:30: >> >>> My hive run out of memory even for a small query: >>> >>> 2022-03-29T20:26:51,440 WARN [Thread-1329] mapred.LocalJobRunner: >>> job_local300585280_0011 >>> >>> java.lang.Exception: java.lang.OutOfMemoryError: Java heap space >>> >>> at >>> org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492) >>> ~[hadoop-mapreduce-client-common-3.3.2.jar:?] >>> >>> at >>> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:552) >>> ~[hadoop-mapreduce-client-common-3.3.2.jar:?] >>> >>> Caused by: java.lang.OutOfMemoryError: Java heap space >>> >>> >>> >>> hadoop-3.3.2 >>> >>> hive-3.1.2 >>> >>> java version "1.8.0_321" >>> >>> >>> >>> How to fix this? thanks. >>> >>> >>> >>> >> >> -- >> ---------------------------------- >> Pau Tallada Crespí >> Departament de Serveis >> Port d'Informació Científica (PIC) >> Tel: +34 93 170 2729 >> ---------------------------------- >> >> -- ---------------------------------- Pau Tallada Crespí Departament de Serveis Port d'Informació Científica (PIC) Tel: +34 93 170 2729 ----------------------------------
