Hi All Having some problems with map reduce running in pseudo-distributed mode. I am running version 1.2.1 on linux. I have: 1. created $JAVA_HOME & $HADOOP_HOME and added the relative bin directories to the path; 2. Formatted the dfs; 3. executed start-dfs.sh and start-mapred.sh.
Executing jps seems to show everything running that should be running (I think). [paul@lt001 bin]$ jps 8724 TaskTracker 8487 SecondaryNameNode 8841 Jps 8353 DataNode 7239 NameNode 8597 JobTracker I have then tried to run the wordcount and pi examples with similar results, eg: [paul@lt001 bin]$ hadoop jar hadoop/hadoop-examples-1.2.1.jar pi 4 1000 Warning: $HADOOP_HOME is deprecated. Number of Maps = 4 Samples per Map = 1000 Wrote input for Map #0 Wrote input for Map #1 Wrote input for Map #2 Wrote input for Map #3 Starting Job 13/11/18 10:31:38 INFO mapred.FileInputFormat: Total input paths to process : 4 13/11/18 10:31:39 INFO mapred.JobClient: Running job: job_201311181028_0001 13/11/18 10:31:40 INFO mapred.JobClient: map 0% reduce 0% 13/11/18 10:31:47 INFO mapred.JobClient: map 50% reduce 0% 13/11/18 10:31:52 INFO mapred.JobClient: map 100% reduce 0% In each instance the output reaches the map 100% reduce 0% stage then stalls. No matter how long I wait the job does not advance any further. I have checked the logs and the one I suspect is indicating the problem is hadoop-paul-tasktracker-lt001.log which has the following output: 2013-11-18 10:31:55,969 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311181028_0001_r_000000_0 0.0% reduce > copy > 2013-11-18 10:34:59,148 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311181028_0001_r_000000_0 0.0% reduce > copy > 2013-11-18 10:35:05,196 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311181028_0001_r_000000_0 0.0% reduce > copy > 2013-11-18 10:35:11,253 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311181028_0001_r_000000_0 0.0% reduce > copy > .......... 2013-11-18 11:10:03,259 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311181028_0001_r_000000_0 0.0% reduce > copy > 2013-11-18 11:10:06,290 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311181028_0001_r_000000_0 0.0% reduce > copy > 2013-11-18 11:10:12,320 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311181028_0001_r_000000_0 0.0% reduce > copy > 2013-11-18 11:10:18,343 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311181028_0001_r_000000_0 0.0% reduce > copy > 2013-11-18 11:10:21,369 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311181028_0001_r_000000_0 0.0% reduce > copy > 2013-11-18 11:10:27,395 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311181028_0001_r_000000_0 0.0% reduce > copy > 2013-11-18 11:10:33,426 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311181028_0001_r_000000_0 0.0% reduce > copy > 2013-11-18 11:10:36,463 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311181028_0001_r_000000_0 0.0% reduce > copy > It seems it is stuck on reduce > copy > but why? Can anyone help with where to look next? Many thanks Paul
