Hi,
I'm really frustrated because I've already lost some days trying to deploy hadoop but it doesn't work. If I deploy in a single cluster all things work ok (the mapreduce example as well the deployment). However when I try to install hadoop in a cluster the problems appear. The configurations are the one that you shown in your blog. After I start the cluster I check the logs and no exceptions are thown. When I start the hadoop-example wordcount the map task is executed without problems ( INFO mapred.JobClient: map 100% reduce 0%) but then stops where and in the console shows "Error reading task outputhttp://master:50060/tasklog?plaintext=true&taskid=attempt_20091228132... <http://www.google.com/url?sa=D&q=http://master:50060/tasklog%3Fplaintext%3Dtrue%26taskid%3Dattempt_200912281326_0001_m_000000_0%26filter%3Dstdout%E2%80%9D&usg=AFQjCNGexaEsrP7sB9Slpcjrxi-wQNWaMA> If I check the log files I can see the following exceptions: In the /userLogs/attemptX 2009-12-28 13:28:36,035 WARN org.apache.hadoop.mapred.ReduceTask: java.io.FileNotFoundException: http://master:50060/mapOutput?job=job_200912281326_0001&map=attempt_2... <http://www.google.com/url?sa=D&q=http://master:50060/mapOutput%3Fjob%3Djob_200912281326_0001%26map%3Dattempt_200912281326_0001_m_000000_0%26reduce%3D0&usg=AFQjCNHUolZY6jZh4toz_Zsx1tISJS8OUA> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at sun.net.www.protocol.http.HttpURLConnection$6.run <http://www.google.com/url?sa=D&q=www.protocol.http.HttpURLConnection%246.run&usg=AFQjCNF3SFlsqNm7hY5we-D3sXnMDeM1cA> (HttpURLConnection.java:1345) at java.security.AccessController.doPrivileged(Native Method) at sun.net.www.protocol.http.HttpURLConnection.getChainedException <http://www.google.com/url?sa=D&q=www.protocol.http.HttpURLConnection.getChainedException&usg=AFQjCNFWiaqz2t5v-uYmsXNAGqk2DoyiTQ> (HttpURLConnection.java:1339) at sun.net.www.protocol.http.HttpURLConnection.getInputStream <http://www.google.com/url?sa=D&q=www.protocol.http.HttpURLConnection.getInputStream&usg=AFQjCNFqUjkHnP5H1HOyzWN1ONhoAxhpCA> (HttpURLConnection.java:993) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier $MapOutputCopier.getInputStream(ReduceTask.java:1447) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier $MapOutputCopier.getMapOutput(ReduceTask.java:1349) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier $MapOutputCopier.copyOutput(ReduceTask.java:1261) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run (ReduceTask.java:1195) Caused by: java.io.FileNotFoundException: http://master:50060/mapOutput?job=job_200912281326_0001&map=attempt_2... <http://www.google.com/url?sa=D&q=http://master:50060/mapOutput%3Fjob%3Djob_200912281326_0001%26map%3Dattempt_200912281326_0001_m_000000_0%26reduce%3D0&usg=AFQjCNHUolZY6jZh4toz_Zsx1tISJS8OUA> at sun.net.www.protocol.http.HttpURLConnection.getInputStream <http://www.google.com/url?sa=D&q=www.protocol.http.HttpURLConnection.getInputStream&usg=AFQjCNFqUjkHnP5H1HOyzWN1ONhoAxhpCA> (HttpURLConnection.java:1288) ... 4 more 2009-12-28 13:28:36,043 INFO org.apache.hadoop.mapred.ReduceTask: Task attempt_200912281326_0001_r_000000_0: Failed fetch #1 from attempt_200912281326_0001_m_000000_0 2009-12-28 13:28:36,043 WARN org.apache.hadoop.mapred.ReduceTask: attempt_200912281326_0001_r_000000_0 adding host master to penalty box, next contact in 4 seconds 2009-12-28 13:28:36,044 INFO org.apache.hadoop.mapred.ReduceTask: attempt_200912281326_0001_r_000000_0: Got 1 map-outputs from previous failures 2009-12-28 13:28:41,044 INFO org.apache.hadoop.mapred.ReduceTask: attempt_200912281326_0001_r_000000_0 Scheduled 1 outputs (0 slow hosts and0 dup hosts) 2009-12-28 13:28:41,054 WARN org.apache.hadoop.mapred.ReduceTask: attempt_200912281326_0001_r_000000_0 copy failed: attempt_200912281326_0001_m_000000_0 from master 2009-12-28 13:28:41,058 WARN org.apache.hadoop.mapred.ReduceTask: java.io.FileNotFoundException: http://master:50060/mapOutput?job=job_200912281326_0001&map=attempt_2... <http://www.google.com/url?sa=D&q=http://master:50060/mapOutput%3Fjob%3Djob_200912281326_0001%26map%3Dattempt_200912281326_0001_m_000000_0%26reduce%3D0&usg=AFQjCNHUolZY6jZh4toz_Zsx1tISJS8OUA> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at sun.net.www.protocol.http.HttpURLConnection$6.run <http://www.google.com/url?sa=D&q=www.protocol.http.HttpURLConnection%246.run&usg=AFQjCNF3SFlsqNm7hY5we-D3sXnMDeM1cA> (HttpURLConnection.java:1345) at java.security.AccessController.doPrivileged(Native Method) at sun.net.www.protocol.http.HttpURLConnection.getChainedException <http://www.google.com/url?sa=D&q=www.protocol.http.HttpURLConnection.getChainedException&usg=AFQjCNFWiaqz2t5v-uYmsXNAGqk2DoyiTQ> (HttpURLConnection.java:1339) at sun.net.www.protocol.http.HttpURLConnection.getInputStream <http://www.google.com/url?sa=D&q=www.protocol.http.HttpURLConnection.getInputStream&usg=AFQjCNFqUjkHnP5H1HOyzWN1ONhoAxhpCA> (HttpURLConnection.java:993) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier $MapOutputCopier.getInputStream(ReduceTask.java:1447) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier $MapOutputCopier.getMapOutput(ReduceTask.java:1349) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier $MapOutputCopier.copyOutput(ReduceTask.java:1261) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run (ReduceTask.java:1195) Caused by: java.io.FileNotFoundException: http://master:50060/mapOutput?job=job_200912281326_0001&map=attempt_2... <http://www.google.com/url?sa=D&q=http://master:50060/mapOutput%3Fjob%3Djob_200912281326_0001%26map%3Dattempt_200912281326_0001_m_000000_0%26reduce%3D0&usg=AFQjCNHUolZY6jZh4toz_Zsx1tISJS8OUA> at sun.net.www.protocol.http.HttpURLConnection.getInputStream <http://www.google.com/url?sa=D&q=www.protocol.http.HttpURLConnection.getInputStream&usg=AFQjCNFqUjkHnP5H1HOyzWN1ONhoAxhpCA> (HttpURLConnection.java:1288) ... 4 more and in the hadoop-root-tasktracker shows 2009-12-28 13:28:34,055 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_200912281326_0001_m_000000_0,0) failed : org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200912281326_0001/ attempt_200912281326_0001_m_000000_0/output/file.out.index in any of the configured local directories at org.apache.hadoop.fs.LocalDirAllocator $AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:389) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead (LocalDirAllocator.java:138) at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet (TaskTracker.java:2886) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: 502) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: 363) at org.mortbay.jetty.security.SecurityHandler.handle (SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: 181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java: 766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: 417) at org.mortbay.jetty.handler.ContextHandlerCollection.handle (ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java: 152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java: 534) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete (HttpConnection.java:864) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403) at org.mortbay.io.nio.SelectChannelEndPoint.run (SelectChannelEndPoint.java:409) at org.mortbay.thread.QueuedThreadPool$PoolThread.run (QueuedThreadPool.java:522) Do you have any idea what could be the problem? I'm a newbie in this, can you tell me if hadoop has a forum or something similar? thanks in advance Hervé
