Re: Accessing Hive Tables in Spark
Seems like Spark can't access hive-site.xml under cluster mode. One solution is to add the config `spark.yarn.dist.files=/path/to/hive-site.xml` to your spark-defaults.conf. And don't forget to call `enableHiveSupport()` on `SparkSession`. Tushar Singhal wrote > Hi Everyone, > > I was accessing Hive Tables in Spark SQL using Scala submitted by > spark-submit command. > When I ran in cluster mode then got error like : Table not found > But the same is working while submitted as client mode. > Please help me to understand why? > > Distribution : Hortonworks > > Thanks in advance !! -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
Re: Accessing Hive Tables in Spark
Applying this fix https://github.com/apache/spark/pull/19663 Or Using --files or --jars /local/path/to/hive-site.xml may works Thanks, Kent -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
Re: Accessing Hive Tables in Spark
Hi Tushar, It seems Spark is not able to access the metastore. It may be because you are using derby metastases which is maintained locally. Please check all your configurations and that Spark has access to the hive-site.xml file with the metastore uri. Thanks, Marco On Tue, 10 Apr 2018, 08:20 Tushar Singhal, wrote: > Hi Everyone, > > I was accessing Hive Tables in Spark SQL using Scala submitted by > spark-submit command. > When I ran in cluster mode then got error like : Table not found > But the same is working while submitted as client mode. > Please help me to understand why? > > Distribution : Hortonworks > > Thanks in advance !! > >
