The problem was occurring because the mapred.job.tracker property in mapred-site.xml file was not set correctly, I changed it and the problem disappeared in a jiffy.
On Wed, Aug 1, 2012 at 1:14 PM, Venky Shankar <[email protected]>wrote: > Can you provide the configuration files: conf/core-site.xml and > conf/mapred-site.xml ? > > -Venky > > On Tue, Jul 31, 2012 at 2:55 PM, ZG Niu <[email protected]> wrote: > >> Hi list, >> >> I see the following exception on release-3.3 configured with hadoop >> compatible storage, >> The problem was occurring when I run the Terasort benchmarks "hadoop >> jar hadoop-0.20.2-examples.jar terasort output sorted" >> >> >> 12/07/31 01:08:36 INFO terasort.TeraSort: starting >> Initializing GlusterFS >> 12/07/31 01:08:36 INFO mapred.FileInputFormat: Total input paths to >> process : 1 >> 12/07/31 01:08:36 WARN snappy.LoadSnappy: Snappy native library is >> available >> 12/07/31 01:08:36 INFO util.NativeCodeLoader: Loaded the native-hadoop >> library >> 12/07/31 01:08:36 INFO snappy.LoadSnappy: Snappy native library loaded >> 12/07/31 01:08:36 INFO zlib.ZlibFactory: Successfully loaded & >> initialized native-zlib library >> 12/07/31 01:08:36 INFO compress.CodecPool: Got brand-new compressor >> Making 1 from 100000 records >> Step size is 100000.0 >> 12/07/31 01:08:36 INFO jvm.JvmMetrics: Initializing JVM Metrics with >> processName=JobTracker, sessionId= >> 12/07/31 01:08:36 INFO mapred.FileInputFormat: Total input paths to >> process : 1 >> 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating >> _partition.lst in >> /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output-work-2689044872239508516 >> with rwxr-xr-x >> 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Cached >> glusterfs://Namenode1:9000/mnt/output/_partition.lst#_partition.lst as >> /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output/_partition.lst >> 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Cached >> glusterfs://Namenode1:9000/mnt/output/_partition.lst#_partition.lst as >> /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output/_partition.lst >> 12/07/31 01:08:37 WARN mapred.LocalJobRunner: LocalJobRunner does not >> support symlinking into current working dir. >> 12/07/31 01:08:37 INFO mapred.TaskRunner: Creating symlink: >> /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output/_partition.lst >> <- /data/mapred/local/localRunner/_partition.lst >> 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating >> symlink: >> /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/job.jar >> <- /data/mapred/local/localRunner/job.jar >> 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating >> symlink: >> /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/.job.xml.crc >> <- /data/mapred/local/localRunner/.job.xml.crc >> 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating >> symlink: >> /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/job.split >> <- /data/mapred/local/localRunner/job.split >> 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating >> symlink: >> /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/job.splitmetainfo >> <- /data/mapred/local/localRunner/job.splitmetainfo >> 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating >> symlink: >> /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/.job.split.crc >> <- /data/mapred/local/localRunner/.job.split.crc >> 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating >> symlink: >> /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/.job.splitmetainfo.crc >> <- /data/mapred/local/localRunner/.job.splitmetainfo.crc >> 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating >> symlink: >> /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/job.xml >> <- /data/mapred/local/localRunner/job.xml >> 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating >> symlink: >> /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/.job.jar.crc >> <- /data/mapred/local/localRunner/.job.jar.crc >> 12/07/31 01:08:37 INFO mapred.JobClient: Running job: job_local_0001 >> 12/07/31 01:08:37 INFO mapred.MapTask: numReduceTasks: 1 >> 12/07/31 01:08:37 INFO mapred.MapTask: io.sort.mb = 100 >> 12/07/31 01:08:37 INFO mapred.MapTask: data buffer = 79691776/99614720 >> 12/07/31 01:08:37 INFO mapred.MapTask: record buffer = 262144/327680 >> 12/07/31 01:08:37 WARN mapred.LocalJobRunner: job_local_0001 >> java.lang.RuntimeException: Error in configuring object >> at >> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) >> at >> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) >> at >> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) >> at >> org.apache.hadoop.mapred.MapTask$OldOutputCollector.<init>(MapTask.java:481) >> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) >> at >> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210) >> Caused by: java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at >> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) >> ... 6 more >> Caused by: java.lang.IllegalArgumentException: can't read paritions file >> at >> org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.configure(TeraSort.java:213) >> ... 11 more >> Caused by: java.io.FileNotFoundException: File _partition.lst does not >> exist. >> at >> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383) >> at >> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251) >> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:776) >> at >> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424) >> at >> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419) >> at >> org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.readPartitions(TeraSort.java:153) >> at >> org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.configure(TeraSort.java:210) >> ... 11 more >> 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Deleted >> path >> /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output/_partition.lst >> 12/07/31 01:08:38 INFO mapred.JobClient: map 0% reduce 0% >> 12/07/31 01:08:38 INFO mapred.JobClient: Job complete: job_local_0001 >> 12/07/31 01:08:38 INFO mapred.JobClient: Counters: 0 >> 12/07/31 01:08:38 INFO mapred.JobClient: Job Failed: NA >> java.io.IOException: Job failed! >> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1246) >> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:248) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >> at >> org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:257) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at >> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) >> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) >> at >> org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at org.apache.hadoop.util.RunJar.main(RunJar.java:186) >> >> >> -- >> Best Regards, >> NiuZG >> >> _______________________________________________ >> Gluster-devel mailing list >> [email protected] >> https://lists.nongnu.org/mailman/listinfo/gluster-devel >> >> > > > -- > Astalavista Baby!!! > -- Best Regards, NiuZG
_______________________________________________ Gluster-devel mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/gluster-devel
