[hadoop] java.lang.NoClassDefFoundError: org/elasticsearch/hadoop/mr/EsOutputFormat

[CAI Longqi]

 

Hello, I’m using elasticsearch-hadoop-2.0.2.jar, and meet the problem:


Exception in thread "main" java.lang.NoClassDefFoundError: 
org/elasticsearch/hadoop/mr/EsOutputFormat

at com.clqb.app.ElasticSearch.run(ElasticSearch.java:46)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

at com.clqb.app.ElasticSearch.main(ElasticSearch.java:60)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

Caused by: java.lang.ClassNotFoundException: 
org.elasticsearch.hadoop.mr.EsOutputFormat

at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at java.lang.ClassLoader.loadClass(ClassLoader.java:425)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

... 9 more


Here’s my program: 


public class ElasticSearch extends Configured implements Tool {

    public static class AwesomeMapper extends Mapper<LongWritable, Text, 
NullWritable, MapWritable> {

        @Override

        protected void map(LongWritable key, Text value, Context 
context) throws IOException, InterruptedException {

            context.write(NullWritable.get(), 
XmlUtils.xmlTextToMapWritable(value)); // XmlUtils is not shown here

        }

    }


    public static class AwesomeReducer extends Reducer<NullWritable, 
MapWritable, NullWritable, NullWritable> {

    }


    public int run(String[] args) throws Exception {

        Configuration conf = getConf();

        conf.set("xmlinput.start", "<page>");

        conf.set("xmlinput.end", "</page>");

        conf.setBoolean("mapred.map.tasks.speculative.execution", false);

        conf.setBoolean("mapred.reduce.tasks.speculative.execution", false);

        conf.set("es.nodes", "localhost:9200");

        conf.set("es.resource", "radio/artists");


        Job job = Job.getInstance(conf);

        job.setJarByClass(ElasticSearch.class);

        job.setInputFormatClass(XmlInputFormat.class);

        job.setOutputFormatClass(EsOutputFormat.class);

        job.setMapOutputValueClass(MapWritable.class);

        job.setMapperClass(AwesomeMapper.class);

        job.setReducerClass(AwesomeReducer.class);


        Path outputPath = new Path(args[1]);

        FileInputFormat.setInputPaths(job, new Path(args[0]));

        FileOutputFormat.setOutputPath(job, outputPath);

        outputPath.getFileSystem(conf).delete(outputPath, true);


        return job.waitForCompletion(true) ? 0 : 1;

    }


    public static void main(String[] args) throws Exception {

        int exitCode = ToolRunner.run(new ElasticSearch(), args);

        System.exit(exitCode);

    }

}


p.s. *I also make sure that I have included 
`elasticsearch-hadoop-2.0.2.jar` in my `-libjars`*. Any suggestions?


Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/762794c8-0bd0-4c16-b1dd-9c914a29a710%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.