Xiaomeng Huang created HIVE-14369:
-------------------------------------
Summary: submit a task with hive on spark to other yarn cluster
failed
Key: HIVE-14369
URL: https://issues.apache.org/jira/browse/HIVE-14369
Project: Hive
Issue Type: Bug
Reporter: Xiaomeng Huang
In our environment, we have two hadoop clusters with HA, named hivecluster and
sparkcluster.
hivecluster is a HA hadoop cluster for hive, which has large hard disk.
sparkcluster is a HA hadoop cluster for spark, which has large memory.
e.g. below is a hdfs-site.xml of hivecluster:
{code}
<property>
<name>dfs.ha.namenodes.hivecluster</name>
<value>nn1,nn2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.hivecluster.nn1</name>
<value>10.17.21.32:9000</value>
</property>
<property>
<name>dfs.namenode.rpc-address.hivecluster.nn2</name>
<value>10.17.21.77:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.hivecluster.nn1</name>
<value>10.17.21.32:50070</value>
</property>
<property>
<name>dfs.namenode.http-address.hivecluster.nn2</name>
<value>10.17.21.77:50070</value>
</property>
{code}
Firstly, I created a hive table located as hdfs://hivecluster/hive/warehouse/xxx
If I use hive on mr, it will run successfully.
But if I use hive on spark to submit a task to yarn cluster of sparkcluster, it
says:
{code}
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.spark.SparkTask
yarn日志显示：
Diagnostics: java.lang.IllegalArgumentException: java.net.UnknownHostException:
hivecluster
Failing this attempt. Failing the application.
{code}
I didn't set host of hivecluster into hdfs-site.xml of sparkcluster
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
