hello
my context : spark, spark-shell 1.0.1 jdk1.7 scala 2.10.4, ES-Hadoop
2.1.0 ( nighly build)
my problem:
- unable to send RDDs from spark to ES
i got a NoClassDefFoundError see below (
org/codehaus/jackson/annotate/JsonClass)
jackson Jars to add to spark shell?
philippe
best regards
--------------------------
$bin/spark-shell --jars
/usr/lib/spark-1.0/lib/elasticsearch-hadoop-2.1.0.jar
......
spark version 1.0.1
Using Scala version 2.10.4
..............
14/08/06 17:19:36 INFO SparkContext: Added JAR
file:/usr/lib/spark-1.0/lib/elasticsearch-hadoop-2.1.0.jar
scala>
import org.elasticsearch.hadoop.mr.EsOutputFormat
import org.elasticsearch.hadoop.mr.EsInputFormat
import org.elasticsearch.hadoop.cfg.ConfigurationOptions
import org.apache.hadoop.mapred.{FileOutputCommitter, FileOutputFormat,
JobConf, OutputFormat}
import org.apache.hadoop.fs.Path
import org.apache.hadoop.io.{MapWritable, Text, NullWritable}
val jobConf = new JobConf(sc.hadoopConfiguration)
jobConf.set("es.resource", "myindex/mytype")
jobConf.set("es.query", "?q=*")
val esRDD = sc.hadoopRDD(jobConf,classOf[EsInputFormat[Text,
MapWritable]],classOf[Text],classOf[MapWritable])
//* up to there everything ok *
es.count()
---->
java.lang.NoClassDefFoundError: org/codehaus/jackson/annotate/JsonClass
at
org.codehaus.jackson.map.introspect.JacksonAnnotationIntrospector.findDeserializationType(JacksonAnnotationIntrospector.java:524)
.........
at
org.elasticsearch.hadoop.rest.RestClient.parseContent(RestClient.java:119)
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f525de7a-2d5f-41d0-b284-9ed2886c8c22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
