Hi Costin, Thanks for support. Well, I'm still experiencing this issue, and for now I see no obvious reasons for it. My only guess is about environment stuff and I'm trying to clean maven dependencies, environment variables, test version compatibility. For the moment nothing had worked.
About the constant, it was to test to ensure my data wasn't corrupted in some way. So I'm pretty sure the exception gives no clue about the real issue. I keep you in touch in case of I discover the reason, may interest somone after all. Aurelien 2014-07-14 18:48 GMT+03:00 Costin Leau <[email protected]>: > Hi, > > Nothing jumps out from your configuration. The error indicates that the > values passed to es-hadoop cannot be processed for some reason. Which is > more surpsing considering your Mapper writes some constants to the output. > I've pushed some improvements to the 2.x branch which explain better > conditions in which the error appears - you can either build the jar > yourself [1] and test it out or wait for the nightly build to publish the > artifact [2]. > > Cheers, > > [1] https://github.com/elasticsearch/elasticsearch-hadoop/tree/2.x > [2] http://build.elasticsearch.com/view/Hadoop/job/es-hadoop-nightly-2x/ > > > On 7/14/14 1:19 PM, Aurélien wrote: > >> Hi, >> >> I can't sort that ! I'm using hadoop CDH3u6, and trying to get ES index >> my data. I tried with raw json and MapWritable, >> I always get the same kind of errors : >> >> | >> >> java.lang.Exception:org.elasticsearch.hadoop. >> EsHadoopIllegalArgumentException:[org.elasticsearch.hadoop. >> serialization.field.MapWritableFieldExtractor@35b5f7bd]cannot >> extract value fromobject[org.apache.hadoop.io.MapWritable@11c757a1] >> at org.apache.hadoop.mapred.LocalJobRunner$Job.run( >> LocalJobRunner.java:349) >> Causedby:org.elasticsearch.hadoop.EsHadoopIllegalArgumentExcepti >> on:[org.elasticsearch.hadoop.serialization.field. >> MapWritableFieldExtractor@35b5f7bd]cannot >> extract value fromobject[org.apache.hadoop.io.MapWritable@11c757a1] >> >> at org.elasticsearch.hadoop.serialization.bulk. >> TemplatedBulk$FieldWriter.write(TemplatedBulk.java:49) >> at org.elasticsearch.hadoop.serialization.bulk. >> TemplatedBulk.writeTemplate(TemplatedBulk.java:101) >> at org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.write( >> TemplatedBulk.java:77) >> at org.elasticsearch.hadoop.rest.RestRepository.writeToIndex( >> RestRepository.java:130) >> at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter. >> write(EsOutputFormat.java:161) >> at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector. >> write(MapTask.java:531) >> at org.apache.hadoop.mapreduce.TaskInputOutputContext.write( >> TaskInputOutputContext.java:80) >> at my.jobs.index.IndexMapper.map(IndexMapper.java:27) >> at my.jobs.index.IndexMapper.map(IndexMapper.java:19) >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:648) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322) >> at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run( >> LocalJobRunner.java:218) >> at java.util.concurrent.Executors$RunnableAdapter. >> call(Executors.java:471) >> at java.util.concurrent.FutureTask$Sync.innerRun( >> FutureTask.java:334) >> at java.util.concurrent.FutureTask.run(FutureTask.java:166) >> at java.util.concurrent.ThreadPoolExecutor.runWorker( >> ThreadPoolExecutor.java:1145) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run( >> ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:724) >> >> | >> >> Seems to me that all is right, here the configuration of the index mapper >> : >> >> | >> Jobjob =newJob(getConf(),"Indexing into Elastic search."); >> >> job.setJarByClass(getClass()); >> DomainRankDriver.loadLibrariesToDistributedCache(job); >> >> Pathinput =newPath(args[0]); >> FileInputFormat.addInputPath(job,input); >> FileOutputFormat.setOutputPath(job,newPath(args[1])); >> >> >> // Used by ES-hadoop to take Text as Json >> job.setOutputFormatClass(EsOutputFormat.class); >> // job.setMapOutputValueClass(Text.class); >> job.setMapOutputValueClass(MapWritable.class); >> job.setMapperClass(IndexMapper.class); >> >> job.setNumReduceTasks(0); >> | >> >> And my simple mapper : >> | >> >> @Override >> publicvoidmap(LongWritablekey,Textvalue,Contextcontext) >> throwsIOException,InterruptedException{ >> MapWritablemap =newMapWritable(); >> map.put(newText("test"),newText("value")); >> context.write(newLongWritable(),map); >> } >> | >> >> Any clue to search for more ? I'm stuck. >> >> Thanks, >> Aurelien >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to >> [email protected] <mailto:elasticsearch+ >> [email protected]>. >> >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/7f6545ab- >> d6d9-4fdf-8923-0b60e0ea5297%40googlegroups.com >> <https://groups.google.com/d/msgid/elasticsearch/7f6545ab- >> d6d9-4fdf-8923-0b60e0ea5297%40googlegroups.com?utm_medium= >> email&utm_source=footer>. >> >> For more options, visit https://groups.google.com/d/optout. >> > -- > Costin > > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/elasticsearch/O1sJ4UQyZNU/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/elasticsearch/53C3FBCE.9040302%40gmail.com. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2B4E3CZSSFWNHeRBQF1FYSBGcX2c_BiXK_vFxg%3D7y%2BL9wZd9nw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
