Re: Is spark-1.6.1-bin-2.6.0 compatible with hive-1.1.0-cdh5.7.1

2016-07-28 Thread Mich Talebzadeh
Ok does it create a derby database and comes back to prompt? For example does spark-sql work OK. If it cannot find the metastore it will create an empty derby database in the same directory and at prompt you can type show databases; and that will only show default! HTH Dr Mich Talebzadeh Lin

Re: Is spark-1.6.1-bin-2.6.0 compatible with hive-1.1.0-cdh5.7.1

2016-07-28 Thread Mohammad Tariq
Hi Mich, Thank you so much for the prompt response! I do have a copy of hive-site.xml in spark conf directory. On Thursday, July 28, 2016, Mich Talebzadeh wrote: > Hi, > > This line > > 2016-07-28 04:36:01,814] INFO Property hive.metastore.integral.jdo.pushdown > unknown - will be ignored (Dat

Re: Is spark-1.6.1-bin-2.6.0 compatible with hive-1.1.0-cdh5.7.1

2016-07-28 Thread Mich Talebzadeh
Hi, This line 2016-07-28 04:36:01,814] INFO Property hive.metastore.integral.jdo.pushdown unknown - will be ignored (DataNucleus.Persistence:77) telling me that you do don't seem to have the softlink to hive-site.xml in $SPARK_HOME/conf hive-site.xml -> /usr/lib/hive/conf/hive-site.xml I sugge

Is spark-1.6.1-bin-2.6.0 compatible with hive-1.1.0-cdh5.7.1

2016-07-28 Thread Mohammad Tariq
Could anyone please help me with this? I have been using the same version of Spark with CDH-5.4.5 successfully so far. However after a recent CDH upgrade I'm not able to run the same Spark SQL module against hive-1.1.0-cdh5.7.1. When I try to run my program Spark tries to connect to local derby Hi