Dear Hadoop User group, This is my virgin post. So please bear with me if I made any mistakes at my early time here.
Recently I successfully downloaded and compiled Hadoop trunk code (svn co http://svn.apache.org/repos/asf/hadoop/common/trunk/ hadoop-trunk) I also deployed the hadoop-3.0.0-SNAPSHOT distribution to a physical cluster of 5 nodes each running 64-bit Cent OS 6. After careful configuration, when I tried running the 1st PI job example: $hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar pi 4 10 The result was successfully calculated but some exception message was also dumped during the execution and I couldn’t figure out why. The exception message goes like this: Starting Job 14/02/20 13:34:12 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 14/02/20 13:34:12 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 14/02/20 13:34:12 INFO input.FileInputFormat: Total input paths to process : 4 14/02/20 13:34:12 INFO mapreduce.JobSubmitter: number of splits:4 14/02/20 13:34:12 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1598491292_0001 14/02/20 13:34:12 INFO mapreduce.Job: The url to track the job: http://localhost:8080/ 14/02/20 13:34:12 INFO mapreduce.Job: Running job: job_local1598491292_0001 14/02/20 13:34:12 INFO mapred.LocalJobRunner: OutputCommitter set in config null 14/02/20 13:34:12 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter 14/02/20 13:34:13 INFO mapred.LocalJobRunner: Waiting for map tasks 14/02/20 13:34:13 INFO mapred.LocalJobRunner: Starting task: attempt_local1598491292_0001_m_000000_0 14/02/20 13:34:13 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 14/02/20 13:34:13 INFO mapred.MapTask: Processing split: hdfs://samdev06:9000/user/hadoop/QuasiMonteCarlo_1392874450186_1650772770/in/part0:0+118 14/02/20 13:34:13 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer 14/02/20 13:34:13 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584) 14/02/20 13:34:13 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100 14/02/20 13:34:13 INFO mapred.MapTask: soft limit at 83886080 14/02/20 13:34:13 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600 14/02/20 13:34:13 INFO mapred.MapTask: kvstart = 26214396; length = 6553600 14/02/20 13:34:13 INFO mapred.LocalJobRunner: 14/02/20 13:34:13 INFO mapred.MapTask: Starting flush of map output 14/02/20 13:34:13 INFO mapred.MapTask: Spilling map output 14/02/20 13:34:13 INFO mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600 14/02/20 13:34:13 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600 14/02/20 13:34:13 INFO mapred.MapTask: Finished spill 0 14/02/20 13:34:13 INFO mapred.Task: Task:attempt_local1598491292_0001_m_000000_0 is done. And is in the process of committing 14/02/20 13:34:13 WARN mapred.Task: Could not find output size java.io.FileNotFoundException: File does not exist: /tmp/hadoop/mapred/local/localRunner/hadoop/jobcache/job_local1598491292_0001/attempt_local1598491292_0001_m_000000_0/output/file.out at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072) at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064) at org.apache.hadoop.mapred.Task.calculateOutputSize(Task.java:1124) at org.apache.hadoop.mapred.Task.sendLastUpdate(Task.java:1102) at org.apache.hadoop.mapred.Task.done(Task.java:1048) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:344) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:244) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Please kindly help me on this. Thanks in advance! Richard
