Having trouble adding external JAR to MapReduce Program

[Jonathan Poon]

Hi Everyone,

I'm running into trouble adding the Avro JAR into my MapReduce program.  I
do the following to try to add the Avro JAR:

export
HADOOP_CLASSPATH="/tmp/singleEvent.jar:/home/jonathanpoon/local/lib/java/avro-1.7.6/avro-mapred-1.7.6-hadoop1.jar:/home/jonathanpoon/local/lib/java/avro-1.7.6/avro-tools-1.7.6.jar:/usr/local/hadoop/hadoop-core-1.2.1.jar"

export
LIBJARS="/tmp/singleEvent.jar,/home/jonathanpoon/local/lib/java/avro-1.7.6/avro-mapred-1.7.6-hadoop1.jar,/home/jonathanpoon/local/lib/java/avro-1.7.6/avro-tools-1.7.6.jar,/usr/local/hadoop/hadoop-core-1.2.1.jar"

hadoop jar AvroReader.jar org.avro.AvroReader -libjars ${LIBJARS}
/user/jonathanpoon/avro /user/jonathanpoon/output

However, I get the following error:

14/02/21 17:01:17 INFO mapred.JobClient: Task Id :
attempt_201402191318_0014_m_000001_2, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.avro.mapreduce.AvroKeyInputFormat
        at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
        at
org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException:
org.apache.avro.mapreduce.AvroKeyInputFormat
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:270)
        at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
        at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
        ... 8 more



Am I placing the Avro JAR files in the improper place?

Thanks for your help!

Jonathan