[
https://issues.apache.org/jira/browse/SPARK-22463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-22463:
------------------------------------
Assignee: (was: Apache Spark)
> Missing hadoop/hive/hbase/etc configuration files in SPARK_CONF_DIR to
> distributed archive
> ------------------------------------------------------------------------------------------
>
> Key: SPARK-22463
> URL: https://issues.apache.org/jira/browse/SPARK-22463
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 2.1.2, 2.2.0
> Reporter: Kent Yao
>
> When I ran self contained sql apps, such as
> {code:java}
> import org.apache.spark.sql.SparkSession
> object ShowHiveTables {
> def main(args: Array[String]): Unit = {
> val spark = SparkSession
> .builder()
> .appName("Show Hive Tables")
> .enableHiveSupport()
> .getOrCreate()
> spark.sql("show tables").show()
> spark.stop()
> }
> }
> {code}
> with **yarn cluster** mode and `hive-site.xml` correctly within
> `$SPARK_HOME/conf`,they failed to connect the right hive metestore for not
> seeing hive-site.xml in AM/Driver's classpath.
> Although submitting them with `--files/--jars local/path/to/hive-site.xml` or
> puting it to `$HADOOP_CONF_DIR/YARN_CONF_DIR` can make these apps works well
> in cluster mode as client mode, according to the official doc, see @
> http://spark.apache.org/docs/latest/sql-programming-guide.html#hive-tables
> > Configuration of Hive is done by placing your hive-site.xml, core-site.xml
> > (for security configuration), and hdfs-site.xml (for HDFS configuration)
> > file in conf/.
> We may respect these configuration files too or modify the doc for
> hive-tables in cluster mode.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
