Hi, I had only one jar on classpath and none in hadoop cluster. I had different types of values in my MapWritable tho. It turns out this was the problem. So I had always Text as a key, but depending on type Text, LongWritable, BooleanWritable or DoubleWritable as value in that map. When I changed everything to be Text it started working.
Is this intended behaviour? Cheers, Kamil. On Friday, December 12, 2014 8:37:03 PM UTC+1, Costin Leau wrote: > > Hi, > > This error is typically tied to a classpath issue - make sure you have > only one elasticsearch-hadoop jar version in your > classpath and on the Hadoop cluster. > > On 12/12/14 5:56 PM, Kamil Dziublinski wrote: > > Hi guys, > > > > I am trying to run a MR job that reads from HDFS and stores into > ElasticSearch cluster. > > > > I am getting following error: > > Error: > org.elasticsearch.hadoop.serialization.EsHadoopSerializationException: > Cannot handle type [class > > org.apache.hadoop.io.MapWritable], instance > [org.apache.hadoop.io.MapWritable@3879429f] using writer > > [org.elasticsearch.hadoop.mr.WritableValueWriter@3fc8f1a2] > > at > org.elasticsearch.hadoop.serialization.builder.ContentBuilder.value(ContentBuilder.java:259) > > > > at > org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.doWriteObject(TemplatedBulk.java:68) > > > > at > org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.write(TemplatedBulk.java:55) > > > > at > org.elasticsearch.hadoop.rest.RestRepository.writeToIndex(RestRepository.java:130) > > > > at > org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:159) > > > > at > org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:635) > > > > at > org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89) > > > > at > org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112) > > > > at > com.teradata.cybershot.mr.es.userprofile.EsOnlineProfileMapper.map(EsOnlineProfileMapper.java:35) > > > > at > com.teradata.cybershot.mr.es.userprofile.EsOnlineProfileMapper.map(EsOnlineProfileMapper.java:20) > > > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) > > at > org.apache.hadoop.mapreduce.lib.input.DelegatingMapper.run(DelegatingMapper.java:55) > > > > at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) > > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:415) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554) > > > > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) > > > > We are using cdh5.1.0 and es-hadoop dependency 2.0.2 > > > > I have this set in my job configuration: > > job.setOutputFormatClass(EsOutputFormat.class); > > job.setMapOutputValueClass(MapWritable.class); > > > > together with nodes and resource props like it is described on ES page. > > > > in my mapper I simply write: context.write(NullWritable.get(), esMap); > where esMap is org.apache.hadoop.io.MapWritable. > > > > I do not know why it's failing as everything looks ok to me. Maybe you > will have some ideas. > > > > Thanks in advance, > > Kamil. > > > > -- > > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to > > [email protected] <javascript:> <mailto: > [email protected] <javascript:>>. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/elasticsearch/71c57e2a-2210-47c0-aa9e-cbbf164ef05b%40googlegroups.com > > > < > https://groups.google.com/d/msgid/elasticsearch/71c57e2a-2210-47c0-aa9e-cbbf164ef05b%40googlegroups.com?utm_medium=email&utm_source=footer>. > > > > For more options, visit https://groups.google.com/d/optout. > > -- > Costin > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/47200ca2-efd7-4741-832d-89c8b9ec088f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
