Can you run this query successfully using hive on mr ?
Best Regards, Jeff Zhang On Wed, Jan 28, 2015 at 10:01 AM, [email protected] <[email protected]> wrote: > > I check the tez document from HDP page > http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.7/bk_installing_manually_book/content/rpm-chap-tez_configure_tez.html. > > tez.am.resource.memory.mb default value is 1536 > My hadoop yarn.app.mapreduce.am.resource.mb value is 5734 MiB > > The configuration mismatch will cause the problem ? > ------------------------------ > [email protected] > > > *发件人：* [email protected] > *发送时间：* 2015-01-27 17:59 > *收件人：* user <[email protected]> > *主题：* 回复: 回复: tez map task and reduce task stay pending forerver > Sorry Gopal V, I made a mistake, My config mapreduce.map.memory.mb is > 2867 . > > ------------------------------ > [email protected] > > > *发件人：* [email protected] > *发送时间：* 2015-01-27 17:58 > *收件人：* user <[email protected]> > *主题：* 回复: 回复: tez map task and reduce task stay pending forerver > Hello Gopal V, > I check my cdh config ,I found mapreduce.map.memory.mb is 2876. > [email protected] > > > *发件人：* [email protected] > *发送时间：* 2015-01-27 17:31 > *收件人：* user <[email protected]> > *主题：* 回复: Re: tez map task and reduce task stay pending forerver > > I check the hivetez.log . No kill request trigger by hive. > ------------------------------ > [email protected] > > > *发件人：* Gopal V <[email protected]> > *发送时间：* 2015-01-27 17:17 > *收件人：* user <[email protected]> > *抄送：* [email protected] > *主题：* Re: tez map task and reduce task stay pending forerver > On 1/27/15, 12:50 AM, [email protected] wrote: > > hive 0.14.0 tez 0.53 hadoop 2.3.0-cdh 5.0.2 > > hive> select * from p_city order by id; > > Query ID = > zhoushugang_20150127163434_da70d957-6ac4-4b8b-a484-42b593838076 > ... > > > -------------------------------------------------------------------------------- > > > > > VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED > > > -------------------------------------------------------------------------------- > > > > > Map 1 INITED 1 0 0 1 0 0 > > Reducer 2 INITED 1 0 0 1 0 0 > > Looks all container requests are pending/unresponsive. > > I see a container request in the log with > > 2015-01-27 15:43:15,434 INFO [TaskSchedulerEventHandlerThread] > rm.YarnTaskSchedulerService: Allocation request for task: > attempt_1419300485749_371785_1_00_000000_0 with request: > Capability[<memory:2867, vCores:3>]Priority[2] host: > yhd-jqhadoop11.int.yihaodian.com rack: null > ... > 2015-01-27 15:43:17,635 INFO [DelayedContainerManager] > rm.YarnTaskSchedulerService: Releasing held container as either there > are pending but unmatched requests or this is not a session, > containerId=container_1419300485749_371785_01_000002, pendingTasks=1, > isSession=true. isNew=true > > That seems to indicate that a container allocation request was made, but > YARN resource manager never responded with a container (or gave the > wrong container?). > > Does the container size 2867 suggest any idea on what that might be? > > Cheers, > Gopal > > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
