Can you try to add those jars in the SPARK_CLASSPATH and give it a try? Thanks Best Regards
On Wed, Jun 24, 2015 at 12:07 AM, Yana Kadiyska <yana.kadiy...@gmail.com> wrote: > Hi folks, I have been using Spark against an external Metastore service > which runs Hive with Cdh 4.6 > > In Spark 1.2, I was able to successfully connect by building with the > following: > > ./make-distribution.sh --tgz -Dhadoop.version=2.0.0-mr1-cdh4.2.0 > -Phive-thriftserver -Phive-0.12.0 > > I see that in Spark 1.4 the Hive 0.12.0 profile is deprecated in favor of > spark.sql.hive.metastore.version/spark.sql.hive.metastore.jars > > When I tried to use this setup spark-shell fails for me with the following > error: > > 15/06/23 18:18:07 INFO hive.HiveContext: Initializing HiveMetastoreConnection > version 0.12.0 using [Ljava.net.URL;@7b7a9a6c > java.lang.ClassNotFoundException: java.lang.NoClassDefFoundError: > com/google/common/base/Preconditions when creating Hive client using > classpath: file:/hive/lib/guava-11.0.2.jar, > file:/hive/lib/hive-exec-0.10.0-cdh4.6.0.jar, > file:/hive/lib/hive-metastore-0.10.0-cdh4.6.0.jar, > file:/hadoop/share/hadoop/mapreduce1/lib/hadoop-common-2.0.0-cdh4.6.0.jar, > file:/hive/lib/commons-logging-1.0.4.jar > > > > I don't know why it's not seeing the class -- it's in the guava jar. If > anyone has had success with 0.12 version please let me know what jars need > to be on the classpath. I think my Hive version might be too outdated but I > don't control the metastore and I had success with Spark1.2 so I'm hoping... > > >
