Nop I have not but I'm glad I'm not the only one :p Le ven. 26 juin 2015 07:54, Tao Li <[email protected]> a écrit :
> Hi Olivier, have you fix this problem now? I still have this fasterxml > NoSuchMethodError. > > 2015-06-18 3:08 GMT+08:00 Olivier Girardot < > [email protected]>: > >> Ok what was wrong was that the spark-env did not contain the >> HADOOP_CONF_DIR properly set to /etc/hadoop/conf/ >> With that fixed, this issue is gone, but I can't seem to get Spark SQL >> 1.4.0 with Hive working on CDH 5.3 or 5.4 : >> Using this command line : >> IPYTHON=1 /.../spark-1.4.0-bin-hadoop2.4/bin/pyspark --master >> yarn-client --driver-class-path `hadoop classpath` >> >> I end up with this issue : >> : java.lang.NoSuchMethodError: >> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer$.handledType()Ljava/lang/Class; >> at >> com.fasterxml.jackson.module.scala.deser.NumberDeserializers$.<init>(ScalaNumberDeserializersModule.scala:49) >> at >> com.fasterxml.jackson.module.scala.deser.NumberDeserializers$.<clinit>(ScalaNumberDeserializersModule.scala) >> at >> com.fasterxml.jackson.module.scala.deser.ScalaNumberDeserializersModule$class.$init$(ScalaNumberDeserializersModule.scala:61) >> at >> com.fasterxml.jackson.module.scala.DefaultScalaModule.<init>(DefaultScalaModule.scala:19) >> at >> com.fasterxml.jackson.module.scala.DefaultScalaModule$.<init>(DefaultScalaModule.scala:35) >> at >> com.fasterxml.jackson.module.scala.DefaultScalaModule$.<clinit>(DefaultScalaModule.scala) >> at >> org.apache.spark.rdd.RDDOperationScope$.<init>(RDDOperationScope.scala:78) >> at >> org.apache.spark.rdd.RDDOperationScope$.<clinit>(RDDOperationScope.scala) >> at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:87) >> at >> org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:118) >> at >> org.apache.spark.sql.execution.Limit.executeCollect(basicOperators.scala:125) >> at org.apache.spark.sql.DataFrame.collect(DataFrame.scala:1255) >> at org.apache.spark.sql.DataFrame.head(DataFrame.scala:1189) >> at org.apache.spark.sql.DataFrame.take(DataFrame.scala:1248) >> at org.apache.spark.sql.DataFrame.showString(DataFrame.scala:176) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231) >> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) >> at py4j.Gateway.invoke(Gateway.java:259) >> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133) >> at py4j.commands.CallCommand.execute(CallCommand.java:79) >> at py4j.GatewayConnection.run(GatewayConnection.java:207) >> at java.lang.Thread.run(Thread.java:745) >> >> This seems to be related to this Jira Issue : >> https://issues.apache.org/jira/browse/SPARK-8332 >> >> This is a blocker for me to deploy a Spark dataframe based app on an >> existing cluster, any input regarding how to create a proper classpath >> would be great. >> >> Regards, >> >> Olivier. >> >> >> Le mer. 17 juin 2015 à 11:37, Olivier Girardot < >> [email protected]> a écrit : >> >>> Hi everyone, >>> After copying the hive-site.xml from a CDH5 cluster, I can't seem to >>> connect to the hive metastore using spark-shell, here's a part of the stack >>> trace I get : >>> >>> 15/06/17 04:41:57 ERROR TSaslTransport: SASL negotiation failure >>> javax.security.sasl.SaslException: GSS initiate failed [Caused by >>> GSSException: No valid credentials provided (Mechanism level: Failed to >>> find any Kerberos tgt)] >>> at >>> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212) >>> at >>> org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) >>> at >>> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253) >>> at >>> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) >>> at >>> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) >>> at >>> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) >>> at java.security.AccessController.doPrivileged(Native Method) >>> >>> The user has a non-expired ticket, I can execute hadoop fs -ls, all in >>> all I should have access to this. >>> I am stuck with this issue on Spark 1.4.0, did not try a version >>> before... >>> >>> Any guess regarding what might be wrong ? >>> >>> Regards, >>> >>> Olivier. >>> >> >
