Re: Spark-hive integration on HDInsight

[amit kumar singh]

Hey jay

How you are making your cluster  are you using spark cluster 

All this thing should be set up automatically 



Sent from my iPhone

> On Feb 21, 2019, at 12:12 PM, Felix Cheung  wrote:
> 
> You should check with HDInsight support
> 
> From: Jay Singh 
> Sent: Wednesday, February 20, 2019 11:43:23 PM
> To: User
> Subject: Spark-hive integration on HDInsight
>  
> I am trying to integrate  spark with hive on HDInsight  spark cluster .
> I copied hive-site.xml in spark/conf directory. In addition I added hive 
> metastore properties like jdbc connection info on Ambari as well. But still 
> the database and tables created using spark-sql are not visible in hive. 
> Changed ‘spark.sql.warehouse.dir’ value also to point to hive warehouse 
> directory.
> Spark does work with hive not having LLAP ON. What am I missing in the 
> configuration to integrate spark with hive ? Any pointer will be appreciated.
>  
> thx  

Re: Spark-hive integration on HDInsight

[Felix Cheung]

You should check with HDInsight support


From: Jay Singh 
Sent: Wednesday, February 20, 2019 11:43:23 PM
To: User
Subject: Spark-hive integration on HDInsight

I am trying to integrate  spark with hive on HDInsight  spark cluster .
I copied hive-site.xml in spark/conf directory. In addition I added hive 
metastore properties like jdbc connection info on Ambari as well. But still the 
database and tables created using spark-sql are not visible in hive. Changed 
‘spark.sql.warehouse.dir’ value also to point to hive warehouse directory.
Spark does work with hive not having LLAP ON. What am I missing in the 
configuration to integrate spark with hive ? Any pointer will be appreciated.

thx

Spark-hive integration on HDInsight

[Jay Singh]

I am trying to integrate  spark with hive on HDInsight  spark cluster .
I copied hive-site.xml in spark/conf directory. In addition I added hive 
metastore properties like jdbc connection info on Ambari as well. But still the 
database and tables created using spark-sql are not visible in hive. Changed 
'spark.sql.warehouse.dir' value also to point to hive warehouse directory.
Spark does work with hive not having LLAP ON. What am I missing in the 
configuration to integrate spark with hive ? Any pointer will be appreciated.

thx