Can you provide the configuration files: conf/core-site.xml and conf/mapred-site.xml ?
-Venky On Tue, Jul 31, 2012 at 2:55 PM, ZG Niu <[email protected]> wrote: > Hi list, > > I see the following exception on release-3.3 configured with hadoop > compatible storage, > The problem was occurring when I run the Terasort benchmarks "hadoop jar > hadoop-0.20.2-examples.jar terasort output sorted" > > > 12/07/31 01:08:36 INFO terasort.TeraSort: starting > Initializing GlusterFS > 12/07/31 01:08:36 INFO mapred.FileInputFormat: Total input paths to > process : 1 > 12/07/31 01:08:36 WARN snappy.LoadSnappy: Snappy native library is > available > 12/07/31 01:08:36 INFO util.NativeCodeLoader: Loaded the native-hadoop > library > 12/07/31 01:08:36 INFO snappy.LoadSnappy: Snappy native library loaded > 12/07/31 01:08:36 INFO zlib.ZlibFactory: Successfully loaded & initialized > native-zlib library > 12/07/31 01:08:36 INFO compress.CodecPool: Got brand-new compressor > Making 1 from 100000 records > Step size is 100000.0 > 12/07/31 01:08:36 INFO jvm.JvmMetrics: Initializing JVM Metrics with > processName=JobTracker, sessionId= > 12/07/31 01:08:36 INFO mapred.FileInputFormat: Total input paths to > process : 1 > 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating > _partition.lst in > /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output-work-2689044872239508516 > with rwxr-xr-x > 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Cached > glusterfs://Namenode1:9000/mnt/output/_partition.lst#_partition.lst as > /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output/_partition.lst > 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Cached > glusterfs://Namenode1:9000/mnt/output/_partition.lst#_partition.lst as > /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output/_partition.lst > 12/07/31 01:08:37 WARN mapred.LocalJobRunner: LocalJobRunner does not > support symlinking into current working dir. > 12/07/31 01:08:37 INFO mapred.TaskRunner: Creating symlink: > /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output/_partition.lst > <- /data/mapred/local/localRunner/_partition.lst > 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating > symlink: > /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/job.jar > <- /data/mapred/local/localRunner/job.jar > 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating > symlink: > /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/.job.xml.crc > <- /data/mapred/local/localRunner/.job.xml.crc > 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating > symlink: > /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/job.split > <- /data/mapred/local/localRunner/job.split > 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating > symlink: > /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/job.splitmetainfo > <- /data/mapred/local/localRunner/job.splitmetainfo > 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating > symlink: > /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/.job.split.crc > <- /data/mapred/local/localRunner/.job.split.crc > 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating > symlink: > /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/.job.splitmetainfo.crc > <- /data/mapred/local/localRunner/.job.splitmetainfo.crc > 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating > symlink: > /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/job.xml > <- /data/mapred/local/localRunner/job.xml > 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating > symlink: > /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/.job.jar.crc > <- /data/mapred/local/localRunner/.job.jar.crc > 12/07/31 01:08:37 INFO mapred.JobClient: Running job: job_local_0001 > 12/07/31 01:08:37 INFO mapred.MapTask: numReduceTasks: 1 > 12/07/31 01:08:37 INFO mapred.MapTask: io.sort.mb = 100 > 12/07/31 01:08:37 INFO mapred.MapTask: data buffer = 79691776/99614720 > 12/07/31 01:08:37 INFO mapred.MapTask: record buffer = 262144/327680 > 12/07/31 01:08:37 WARN mapred.LocalJobRunner: job_local_0001 > java.lang.RuntimeException: Error in configuring object > at > org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) > at > org.apache.hadoop.mapred.MapTask$OldOutputCollector.<init>(MapTask.java:481) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) > ... 6 more > Caused by: java.lang.IllegalArgumentException: can't read paritions file > at > org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.configure(TeraSort.java:213) > ... 11 more > Caused by: java.io.FileNotFoundException: File _partition.lst does not > exist. > at > org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383) > at > org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251) > at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:776) > at > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424) > at > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419) > at > org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.readPartitions(TeraSort.java:153) > at > org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.configure(TeraSort.java:210) > ... 11 more > 12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Deleted > path > /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output/_partition.lst > 12/07/31 01:08:38 INFO mapred.JobClient: map 0% reduce 0% > 12/07/31 01:08:38 INFO mapred.JobClient: Job complete: job_local_0001 > 12/07/31 01:08:38 INFO mapred.JobClient: Counters: 0 > 12/07/31 01:08:38 INFO mapred.JobClient: Job Failed: NA > java.io.IOException: Job failed! > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1246) > at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:248) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:257) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) > at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) > at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:186) > > > -- > Best Regards, > NiuZG > > _______________________________________________ > Gluster-devel mailing list > [email protected] > https://lists.nongnu.org/mailman/listinfo/gluster-devel > > -- Astalavista Baby!!!
_______________________________________________ Gluster-devel mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/gluster-devel
