That exception looks like one that I see when using too new of a version of
Jackson. See https://issues.apache.org/jira/browse/SPARK-8332. The short
answer is that you probably need to make sure you are using Jackson 2.4.x
rather than 2.5.x.
~ Jonathan
On Fri, Oct 16, 2015 at 7:37 AM, Armand Naude <armandna...@gmail.com> wrote:
> I have setup a standalone single node "cluster" running the following:
>
> - Cassandra 2.2.2
> - Spark 1.5.1
> - List item
> - Compiled fat jar for Spark-Cassandra-Connector 1.5.0-M2
> - Compiled Zeppelin 0.6 snapshot compiled with: mvn -Pspark-1.5
> -Dspark.version=1.5.1 -Dhadoop.version=2.6.0 -Phadoop-2.4 -DskipTests clean
> package
>
> I can work perfectly fine with spark shell retrieving data from cassandra
>
> I have altered the Zeppelin-env.sh as follow:
>
> export MASTER=spark://localhost:7077
> export SPARK_HOME=/root/spark-1.5.1-bin-hadoop2.6/
> export ZEPPELIN_PORT=8880
> export
> ZEPPELIN_JAVA_OPTS="-Dspark.jars=/opt/sparkconnector/spark-cassandra-connector-assembly-1.5.0-M2-SNAPSHOT.jar
> -Dspark.cassandra.connection.host=localhost"
> export ZEPPELIN_NOTEBOOK_DIR="/root/gowalla-spark-demo/notebooks/zeppelin"
> export SPARK_SUBMIT_OPTIONS="--jars
> /opt/sparkconnector/spark-cassandra-connector-assembly-1.5.0-M2-SNAPSHOT.jar
> --deploy-mode cluster"
> export ZEPPELIN_INTP_JAVA_OPTS=$ZEPPELIN_JAVA_OPTS
>
> I then start adding paragraphs to a notebook and import the following
> first:
>
> import com.datastax.spark.connector._
> import com.datastax.spark.connector.cql._
> import com.datastax.spark.connector.rdd.CassandraRDD
> import org.apache.spark.rdd.RDD
> import org.apache.spark.SparkContext
> import org.apache.spark.SparkConf
>
> Not sure if all of these are necessary. This paragraph runs fine.
>
> Then I do the following:
>
> val checkins = sc.cassandraTable("lbsn", "checkins")
>
> This runs fine and returns:
>
> checkins:
> com.datastax.spark.connector.rdd.CassandraTableScanRDD[com.datastax.spark.connector.CassandraRow]
> = CassandraTableScanRDD[0] at RDD at CassandraRDD.scala:15
>
> Then the next paragraph - the follow 2 statements are run -the first
> succeeds and the second fails:
>
> checkins.count
> checkins.first
>
> Result:
>
> res13: Long = 138449
> com.fasterxml.jackson.databind.JsonMappingException: Could not find creator
> property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope)
> at [Source: {"id":"4","name":"first"}; line: 1, column: 1]
> at
> com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
> at
> com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:843)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.addBeanProps(BeanDeserializerFactory.java:533)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:220)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:143)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:409)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:358)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:265)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:245)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:143)
> at
> com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:439)
> at
> com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3666)
> at
> com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3558)
> at
> com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2578)
> at
> org.apache.spark.rdd.RDDOperationScope$.fromJson(RDDOperationScope.scala:82)
> at org.apache.spark.rdd.RDD$$anonfun$34.apply(RDD.scala:1582)
> at org.apache.spark.rdd.RDD$$anonfun$34.apply(RDD.scala:1582)
> at scala.Option.map(Option.scala:145)
> at org.apache.spark.rdd.RDD.<init>(RDD.scala:1582)
> at com.datastax.spark.connector.rdd.CassandraRDD.<init>(CassandraRDD.scala:15)
> at
> com.datastax.spark.connector.rdd.CassandraTableScanRDD.<init>(CassandraTableScanRDD.scala:59)
> at
> com.datastax.spark.connector.rdd.CassandraTableScanRDD.copy(CassandraTableScanRDD.scala:92)
> at
> com.datastax.spark.connector.rdd.CassandraTableScanRDD.copy(CassandraTableScanRDD.scala:59)
> at com.datastax.spark.connector.rdd.CassandraRDD.limit(CassandraRDD.scala:103)
> at com.datastax.spark.connector.rdd.CassandraRDD.take(CassandraRDD.scala:122)
> at org.apache.spark.rdd.RDD$$anonfun$first$1.apply(RDD.scala:1312)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:306)
> at org.apache.spark.rdd.RDD.first(RDD.scala:1311)
> at
> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:36)
> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:41)
> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:43)
> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:45)
> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:47)
> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:49)
> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:51)
> at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:53)
> at $iwC$$iwC$$iwC$$iwC.<init>(<console>:55)
> at $iwC$$iwC$$iwC.<init>(<console>:57)
> at $iwC$$iwC.<init>(<console>:59)
> at $iwC.<init>(<console>:61)
> at <init>(<console>:63)
> at .<init>(<console>:67)
> at .<clinit>(<console>)
> at .<init>(<console>:7)
> at .<clinit>(<console>)
> at $print(<console>)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
> at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1340)
> at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
> at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
> at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
> at
> org.apache.zeppelin.spark.SparkInterpreter.interpretInput(SparkInterpreter.java:655)
> at
> org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:620)
> at
> org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:613)
> at
> org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
> at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:276)
> at org.apache.zeppelin.scheduler.Job.run(Job.java:170)
> at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:118)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> Why does the call to first fail. Calls such as sc.fromTextFile also fail.
>
> the following also works:
>
> checkins.where("year = 2010 and month=2 and day>12 and day<15").count()
>
> but this does not:
>
> checkins.where("year = 2010 and month=2 and day>12 and day<15").first()
>
> Please assist as this is driving me insane. Especially since the spark
> shell works but this does not or at least seems partially broken.
>
> Thanks
> --
> Armand Naudé
> Software Engineer
>
> Cell: +27 082 552 0421
>
> ------------------------------------------------------------
> To see a world in a grain of sand
> And a heaven in a wild flower,
> Hold infinity in the palm of your hand
> And eternity in an hour.
> -- William Blake
>
