Hello everybody, I’m sorry if this has already been covered somewhere else, but I’ve been searching the web for weeks to no avail. :(
Anyway, I am attempting to set up a single-node cluster following the directions here: http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Single-Node_Cluster) . I get everything set up fine and attempt to start the first example program (running the wordcount job). I format the namenode, start-all, and copy the files from local. I then try to execute the example job, but this is the output: [EMAIL PROTECTED] hadoop-0.18.1]$ bin/hadoop jar hadoop-0.18.1-examples.jar wordcoun t books booksOutput 08/11/13 18:21:41 INFO mapred.FileInputFormat: Total input paths to process : 3 08/11/13 18:21:41 INFO mapred.FileInputFormat: Total input paths to process : 3 08/11/13 18:21:42 INFO mapred.JobClient: Running job: job_200811131821_0001 08/11/13 18:21:43 INFO mapred.JobClient: map 0% reduce 0% 08/11/13 18:21:49 INFO mapred.JobClient: map 66% reduce 0% 08/11/13 18:21:52 INFO mapred.JobClient: map 100% reduce 0% 08/11/13 18:21:52 INFO mapred.JobClient: Task Id : attempt_200811131821_0001_m_000001_0, Status : FAILED Map output lost, rescheduling: getMapOutput(attempt_200811131821_0001_m_000001_0,0) failed : org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200811131821_0001/attempt_200811131821_0001_m_000001_0/output/file.out.index in any of the configured local directories at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:359) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138) at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:2402) at javax.servlet.http.HttpServlet.service(HttpServlet.java:689) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:427) at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:475) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567) at org.mortbay.http.HttpContext.handle(HttpContext.java:1565) at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:635) at org.mortbay.http.HttpContext.handle(HttpContext.java:1517) at org.mortbay.http.HttpServer.service(HttpServer.java:954) at org.mortbay.http.HttpConnection.service(HttpConnection.java:814) at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:981) at org.mortbay.http.HttpConnection.handle(HttpConnection.java:831) at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244) at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357) at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534): So apparently the map output is being lost, and reduce can not find it. Upon viewing the logs, I also find this error in the secondary namenode log file: 2008-11-13 17:41:40,518 WARN org.mortbay.jetty.servlet.WebApplicationContext: Configuration error on file:/home/staff/hadoop/hadoop-0.18.1/webapps/secondary java.io.FileNotFoundException: file:/home/staff/hadoop/hadoop-0.18.1/webapps/secondary at org.mortbay.jetty.servlet.WebApplicationContext.resolveWebApp(WebApplicationContext.java:266) at org.mortbay.jetty.servlet.WebApplicationContext.doStart(WebApplicationContext.java:449) at org.mortbay.util.Container.start(Container.java:72) at org.mortbay.http.HttpServer.doStart(HttpServer.java:753) at org.mortbay.util.Container.start(Container.java:72) at org.apache.hadoop.mapred.StatusHttpServer.start(StatusHttpServer.java:207) at org.apache.hadoop.dfs.SecondaryNameNode.initialize(SecondaryNameNode.java:156) at org.apache.hadoop.dfs.SecondaryNameNode.<init>(SecondaryNameNode.java:108) at org.apache.hadoop.dfs.SecondaryNameNode.main(SecondaryNameNode.java:460) I have the following defined in hadoop-site: <property> <name>hadoop.tmp.dir</name> <value>tmp</value> <description>A base for other temporary directories</description> </property> <property> <name>fs.default.name</name> <value>hdfs://localhost:54310</value> <description>The name of the default file system. A URI whose scheme and authority determine the FileSystem implementation. The uri's scheme determines the config property (fs.SCHEME.impl) naming the FileSystem implementation class. The uri's authority is used to determine the host, port, etc. for a filesystem.</description> </property> <property> <name>mapred.job.tracker</name> <value>localhost:54311</value> <description>The host and port that the MapReduce job tracker runs at. If "local", then jobs are run in-process as a single map and reduce task. </description> </property> <property> <name>dfs.replication</name> <value>1</value> <description>Default block replication. The actual number of replications can be specified when the file is created. The default is used if replication is not specified in create time. </description> </property> <property> <name>mapred.local.dir</name> <value>stores</value> <description>The local directory where MapReduce stores intermediate data files. May be a comma-separated list of directories on different devices in order to spread disk i/o. Directories that do not exist are ignored. </description> </property> Does any of this look familiar to anybody? My ultimate goal is to have a cluster running on 20 or so linux machines. I thought that running a single-node cluster would be a good start, but so far I’ve been tackling error after error for the past few weeks. I appreciate any help anybody can give me! - Kevin Workman -- View this message in context: http://www.nabble.com/Could-Not-Find-file.out.index-%28Help-starting-Hadoop%21%29-tp20492507p20492507.html Sent from the Hadoop core-user mailing list archive at Nabble.com.
