Hi,
What version of es-hadoop are you using? The problem stems from the difference in the types mentioned on your Mapper,
namely the output value class:
conf.setMapOutputValueClass(MapWritable.class);
to MapWritable while LinkedMapWritable is returned. The latest versions automatically detect this and use the proper
type so I recommend upgrading.
If that's not an option, use LinkedMapWritable.
Cheers,
P.S. Please don't post code and stracktraces on the mailing list since it highly reduces the readability of your email.
Instead use gist or any other service
to post the code as indicated in the docs [1]. Thanks
[1]
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/troubleshooting.html#_where_do_i_post_my_information
On 9/9/14 11:59 AM, gaurav redkar wrote:
Hi, I was following the example given on official elasticsearch documentation
to read data from elasticsearch using
hadoop but i am getting the following error.
java.lang.Exception: java.io.IOException: Type mismatch in value from map:
expected org.apache.hadoop.io.MapWritable,
received org.elasticsearch.hadoop.mr.LinkedMapWritable
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.io.IOException: Type mismatch in value from map: expected
org.apache.hadoop.io.MapWritable, received
org.elasticsearch.hadoop.mr.LinkedMapWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1054)
at org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:587)
at es2.mapper1.map(mapper1.java:29)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
14/09/09 13:25:51 INFO mapreduce.Job: Job job_local994189535_0001 failed with
state FAILED due to: NA
14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836)
at es2.Es2.main(Es2.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
14/09/09 13:25:51 INFO mapred.LocalJobRunner: ShardInputSplit
[node=[QMYbfSRMTsyXVVgfFweMkA/Conceptnet|192.168.1.202:9200],shard=0] > map
The code for my main runner class is as follows:-
public class Es2 {
static private final Path TMP_DIR = new Path(Es2.class.getSimpleName()
+ "_TMP_1");
/**
* @param args the command line arguments
*/
public static void main(String[] args) throws IOException{
//#String node="192.168.1.202";
JobConf conf = new JobConf();
conf.set("es.resource", "data/content");
conf.set("es.nodes", "192.168.1.202:9200");
conf.set("es.query", "?q=s_start:java");
conf.setInputFormat(EsInputFormat.class);
conf.setMapOutputKeyClass(Text.class);
conf.setMapOutputValueClass(MapWritable.class);
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(MapWritable.class);
conf.setOutputFormat(TextOutputFormat.class);
conf.setMapperClass(mapper1.class);
final Path outDir = new Path(TMP_DIR, "out");
FileOutputFormat.setOutputPath(conf, outDir);
JobClient.runJob(conf);
}
}
The code for my mapper class is as follows:-
public class mapper1 extends MapReduceBase implements
Mapper{
@Override
public void map(Object key, Object value, OutputCollector output,
Reporter reporter) throws IOException {
Text docId = (Text) key;
MapWritable doc = (MapWritable) value;
output.collect(docId,doc);
}
}
Kindly guide me on this issue.
Thanks
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
[email protected]
<mailto:[email protected]>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
Costin
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/540ED720.5040501%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
