Hello Hive users, I am attempting to execute a reduce job against a Yarn enabled Hadoop cluster. Plain map jobs run without error. I am able to use the Hadoop streaming jars to execute simple map and reduce jobs, so it seems that something must be wrong (most likely misconfigured) with my Hive integration.
Package versions: hive-0.8.1+61-1.cdh4.0.1.p0.1.el6.noarch hadoop-2.0.0+91-1.cdh4.0.1.p0.1.el6.x86_64 hadoop-hdfs-secondarynamenode-2.0.0+91-1.cdh4.0.1.p0.1.el6.x86_64 hadoop-yarn-proxyserver-2.0.0+91-1.cdh4.0.1.p0.1.el6.x86_64 hadoop-yarn-2.0.0+91-1.cdh4.0.1.p0.1.el6.x86_64 hadoop-hdfs-2.0.0+91-1.cdh4.0.1.p0.1.el6.x86_64 hadoop-0.20-mapreduce-0.20.2+1216-1.cdh4.0.1.p0.1.el6.x86_64 hadoop-client-2.0.0+91-1.cdh4.0.1.p0.1.el6.x86_64 hadoop-mapreduce-historyserver-2.0.0+91-1.cdh4.0.1.p0.1.el6.x86_64 hadoop-yarn-resourcemanager-2.0.0+91-1.cdh4.0.1.p0.1.el6.x86_64 hadoop-mapreduce-2.0.0+91-1.cdh4.0.1.p0.1.el6.x86_64 The error output is only the process registering that it cannot log it's job output with the JobHistory server (I think this is known issue with Hive / Yarn?). Here is a link to pastebin with the full error output: http://pastebin.com/DH9auij7 My yarn-site.xml and hive-site.xml are here:http://pastebin.com/LK3rnrgb A side note: if I set HADOOP_MAPRED_HOME=/usr/lib/hadoop-mapreduce, the hive M/R process will hang indefinitely outputting: 2012-08-02 15:15:22,032 null map = 0%, reduce = 0% Any assistance would be greatly appreciated. This is a test cluster, so we can pursue any troubleshooting steps that may illuminate the issue. Regards, Joseph Hammerman
