Hello,
When I start a map-reduce job, it starts, and after a short while,
fails with the error below (SnappyCodec not found).
I am currently starting the job from other Java code (so the Hadoop
executable in the bin directory is not used anymore), but in principle
this seems to work (in the admin of the Jobtracker the job shows up
when it starts). However after a short while the map task fails with:
java.lang.IllegalArgumentException: Compression codec
org.apache.hadoop.io.compress.SnappyCodec not found.
at
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:96)
at
org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:134)
at
org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:62)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:522)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.io.compress.SnappyCodec
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:334)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
at
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:89)
... 10 more
I confirmed that the SnappyCodec class is present in the
hadoop-core-1.0.2.jar, and the snappy-java-1.0.4.1.jar is present as
well. The directory of those jars is on the HADOOP_CLASSPATH, but it
seems it still cannot find it. I also checked that the config files of
Hadoop are read. I run all nodes on localhost.
Any suggestions on what could be the cause of the issue?
Regards,
Bas
