Hi, I was following the example given on official elasticsearch
documentation to read data from elasticsearch using hadoop but i am getting
the following error.
java.lang.Exception: java.io.IOException: Type mismatch in value from map:
expected org.apache.hadoop.io.MapWritable, received
org.elasticsearch.hadoop.mr.LinkedMapWritable
at
org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.io.IOException: Type mismatch in value from map: expected
org.apache.hadoop.io.MapWritable, received
org.elasticsearch.hadoop.mr.LinkedMapWritable
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1054)
at
org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:587)
at es2.mapper1.map(mapper1.java:29)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
14/09/09 13:25:51 INFO mapreduce.Job: Job job_local994189535_0001 failed
with state FAILED due to: NA
14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836)
at es2.Es2.main(Es2.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
14/09/09 13:25:51 INFO mapred.LocalJobRunner: ShardInputSplit
[node=[QMYbfSRMTsyXVVgfFweMkA/Conceptnet|192.168.1.202:9200],shard=0] > map
The code for my main runner class is as follows:-
public class Es2 {
static private final Path TMP_DIR = new
Path(Es2.class.getSimpleName()
+ "_TMP_1");
/**
* @param args the command line arguments
*/
public static void main(String[] args) throws IOException{
//#String node="192.168.1.202";
JobConf conf = new JobConf();
conf.set("es.resource", "data/content");
conf.set("es.nodes", "192.168.1.202:9200");
conf.set("es.query", "?q=s_start:java");
conf.setInputFormat(EsInputFormat.class);
conf.setMapOutputKeyClass(Text.class);
conf.setMapOutputValueClass(MapWritable.class);
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(MapWritable.class);
conf.setOutputFormat(TextOutputFormat.class);
conf.setMapperClass(mapper1.class);
final Path outDir = new Path(TMP_DIR, "out");
FileOutputFormat.setOutputPath(conf, outDir);
JobClient.runJob(conf);
}
}
The code for my mapper class is as follows:-
public class mapper1 extends MapReduceBase implements
Mapper{
@Override
public void map(Object key, Object value, OutputCollector output,
Reporter reporter) throws IOException {
Text docId = (Text) key;
MapWritable doc = (MapWritable) value;
output.collect(docId,doc);
}
}
Kindly guide me on this issue.
Thanks
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
