Hi Phil,
Glad to see the work in es-hadoop master is being picked up even without any
public announcement of it :)
The issue has been fixed in master [1] and already pushed to Maven - can you
please update and try again?
FTR: The issue seems to be caused by multiple versions of Jackson which are pulled in the classpath (one from Hadoop,
another from Spark)
which on some platforms, causes class loading issues in Jackson during
start-up. The fix in master hopefully remedies that.
Cheers,
[1] https://github.com/elasticsearch/elasticsearch-hadoop/issues/239
On 8/6/14 7:32 PM, Phil gib wrote:
hello
my context : spark, spark-shell 1.0.1 jdk1.7 scala 2.10.4, ES-Hadoop 2.1.0 (
nighly build)
my problem:
- unable to send RDDs from spark to ES
i got a NoClassDefFoundError see below (
org/codehaus/jackson/annotate/JsonClass)
jackson Jars to add to spark shell?
philippe
best regards
--------------------------
$bin/spark-shell --jars /usr/lib/spark-1.0/lib/elasticsearch-hadoop-2.1.0.jar
......
spark version 1.0.1
Using Scala version 2.10.4
..............
14/08/06 17:19:36 INFO SparkContext: Added JAR
file:/usr/lib/spark-1.0/lib/elasticsearch-hadoop-2.1.0.jar
scala>
import org.elasticsearch.hadoop.mr.EsOutputFormat
import org.elasticsearch.hadoop.mr.EsInputFormat
import org.elasticsearch.hadoop.cfg.ConfigurationOptions
import org.apache.hadoop.mapred.{FileOutputCommitter, FileOutputFormat,
JobConf, OutputFormat}
import org.apache.hadoop.fs.Path
import org.apache.hadoop.io.{MapWritable, Text, NullWritable}
val jobConf = new JobConf(sc.hadoopConfiguration) jobConf.set("es.resource",
"myindex/mytype")
jobConf.set("es.query", "?q=*")
val esRDD = sc.hadoopRDD(jobConf,classOf[EsInputFormat[Text,
MapWritable]],classOf[Text],classOf[MapWritable])
//*up to there everything ok *
es.count()
---->
java.lang.NoClassDefFoundError: org/codehaus/jackson/annotate/JsonClass
at
org.codehaus.jackson.map.introspect.JacksonAnnotationIntrospector.findDeserializationType(JacksonAnnotationIntrospector.java:524)
.........
at
org.elasticsearch.hadoop.rest.RestClient.parseContent(RestClient.java:119)
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
[email protected]
<mailto:[email protected]>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f525de7a-2d5f-41d0-b284-9ed2886c8c22%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/f525de7a-2d5f-41d0-b284-9ed2886c8c22%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
Costin
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/53E25BF0.4000307%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
