"HADOOP-1374" refers to the JIRA (Hadoop's issue-tracking software of choice) ticket that details a bug that may be related to yours. Check it out here: http://issues.apache.org/jira/browse/HADOOP-1374.
Regards, Jeff On 8/2/07, Samuel LEMOINE <[EMAIL PROTECTED]> wrote: > > Thanks for your answer, you give me hope :) > TO answer your interrogations, I use the haddop script "start-all.sh" to > launch hadoop, and each plateform is a JAVA 1.5.0 jdk running on ubuntu > 7.0.4. > > Your few advices raise as many questions for me : you mention a > discussion named "HADOOP-1374", what does it refer to ? I've crawled in > the old mail archives with the keyword "1374" and found some messages > about the tasktracker error report number 1374, I guess it's what you > was talking about. If it's the case, where should I see this error? (the > tasktracker's logs don't seem to print any error) > Then, you tell me to try changing the order in which the slaves start : > I was using the start-all.sh so that I had no controle on that, but even > with a different starting order (by the use of bin/hadoop datanode and > bin/hadoop tasktracker on each slave), it doesn't solve the problem, > still blocking during the reduce phase. > > I tryed to kill tasktrackers, but I am not really able to determine > which one I have to kill, actualy they both look the same, only the % > changes. I managed to finish a job by killing tasktrackers and > restarting them, one after one, progressing step by step, but > actually... it's like doing the job with only one slave, and it already > worked before. There seems to get a serious problem with my reduce task > when running on 2 tasktrackers (or more, I'm actually running on 3 > tasktrackers now, as I thought it could unblock the scheme, but it > didn't). > > My last "more successful" attempt is not very encouraging, but I'll > expose it in a more detailed way: I dispose of 4 plateforms (each one > running with a 1.5.0 jdk under 7.0.4 ubuntu), I'll call them 90, 91, 102 > and 131 (their DHCP attributed ip actually). 131 is my coputer, the 3 > other are 3 virtual machines. > 131> bin/hadoop namenode -format > 131> bin/hadoop namenode > 131> bin/hadoop dfs -put /.../documents /user/hadoop/documents > 90> bin/hadoop datanode > 91> bin/hadoop datanode > 102> bin/hadoop datanode > 131> bin/hadoop jobtracker > 90> bin/hadoop tasktracker > 91> bin/hadoop tasktracker > 102> bin/hadoop tasktracker > 131> bin/hadoop --config conf/ jar hadoop-0.13.1-examples.jar grep > /user/hadoop/documents /user/hadoop/results blabla > > and here is what it gives: > 07/08/02 09:20:04 INFO mapred.FileInputFormat: Total input paths to > process : 7 > 07/08/02 09:20:04 INFO mapred.JobClient: Running job: job_0001 > 07/08/02 09:20:05 INFO mapred.JobClient: map 0% reduce 0% > 07/08/02 09:20:09 INFO mapred.JobClient: map 25% reduce 0% > 07/08/02 09:20:11 INFO mapred.JobClient: map 50% reduce 0% > 07/08/02 09:20:12 INFO mapred.JobClient: map 75% reduce 0% > 07/08/02 09:20:14 INFO mapred.JobClient: map 100% reduce 0% > 07/08/02 09:20:22 INFO mapred.JobClient: map 100% reduce 11% > 07/08/02 09:21:03 INFO mapred.JobClient: map 100% reduce 4% > 07/08/02 09:21:03 INFO mapred.JobClient: Task Id : task_0001_m_000001_0, > Status : FAILED > 07/08/02 09:21:03 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:21:03 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:21:03 INFO mapred.JobClient: Task Id : task_0001_m_000002_0, > Status : FAILED > 07/08/02 09:21:03 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:21:03 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:21:03 INFO mapred.JobClient: Task Id : task_0001_m_000003_0, > Status : FAILED > 07/08/02 09:21:03 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:21:03 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:21:03 INFO mapred.JobClient: Task Id : task_0001_m_000004_0, > Status : FAILED > 07/08/02 09:21:03 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:21:03 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:21:03 INFO mapred.JobClient: Task Id : task_0001_m_000005_0, > Status : FAILED > 07/08/02 09:21:03 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:21:03 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:21:03 INFO mapred.JobClient: Task Id : task_0001_r_000001_0, > Status : FAILED > 07/08/02 09:21:03 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:21:03 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:21:04 INFO mapred.JobClient: map 87% reduce 4% > 07/08/02 09:21:07 INFO mapred.JobClient: map 100% reduce 4% > 07/08/02 09:21:16 INFO mapred.JobClient: map 100% reduce 11% > 07/08/02 09:21:53 INFO mapred.JobClient: map 100% reduce 6% > 07/08/02 09:21:53 INFO mapred.JobClient: Task Id : task_0001_m_000001_1, > Status : FAILED > 07/08/02 09:21:53 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:21:53 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:21:53 INFO mapred.JobClient: Task Id : task_0001_m_000002_1, > Status : FAILED > 07/08/02 09:21:53 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:21:53 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:21:53 INFO mapred.JobClient: Task Id : task_0001_m_000004_1, > Status : FAILED > 07/08/02 09:21:53 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:21:53 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:21:53 INFO mapred.JobClient: Task Id : task_0001_r_000002_0, > Status : FAILED > 07/08/02 09:21:53 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:21:53 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:22:07 INFO mapred.JobClient: map 100% reduce 33% > 07/08/02 09:22:48 INFO mapred.JobClient: Task Id : task_0001_r_000002_1, > Status : FAILED > 07/08/02 09:22:48 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:22:48 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:22:58 INFO mapred.JobClient: map 100% reduce 66% > 07/08/02 09:23:17 INFO mapred.JobClient: Task Id : task_0001_r_000001_1, > Status : FAILED > 07/08/02 09:23:17 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:23:17 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:23:26 INFO mapred.JobClient: map 100% reduce 100% > 07/08/02 09:23:27 INFO mapred.JobClient: Job complete: job_0001 > 07/08/02 09:23:27 INFO mapred.JobClient: Counters: 9 > 07/08/02 09:23:27 INFO mapred.JobClient: Map-Reduce Framework > 07/08/02 09:23:27 INFO mapred.JobClient: Map input records=12 > 07/08/02 09:23:27 INFO mapred.JobClient: Map output records=2 > 07/08/02 09:23:27 INFO mapred.JobClient: Map input bytes=84 > 07/08/02 09:23:27 INFO mapred.JobClient: Map output bytes=30 > 07/08/02 09:23:27 INFO mapred.JobClient: Combine input records=2 > 07/08/02 09:23:27 INFO mapred.JobClient: Combine output records=2 > 07/08/02 09:23:27 INFO mapred.JobClient: Reduce input groups=1 > 07/08/02 09:23:27 INFO mapred.JobClient: Reduce input records=2 > 07/08/02 09:23:27 INFO mapred.JobClient: Reduce output records=1 > 07/08/02 09:23:27 INFO mapred.FileInputFormat: Total input paths to > process : 3 > 07/08/02 09:23:28 INFO mapred.JobClient: Running job: job_0002 > 07/08/02 09:23:29 INFO mapred.JobClient: map 0% reduce 0% > 07/08/02 09:23:37 INFO mapred.JobClient: map 33% reduce 0% > 07/08/02 09:23:38 INFO mapred.JobClient: map 66% reduce 0% > 07/08/02 09:23:39 INFO mapred.JobClient: map 100% reduce 0% > 07/08/02 09:23:47 INFO mapred.JobClient: map 100% reduce 11% > 07/08/02 09:24:04 INFO mapred.JobClient: Task Id : task_0002_r_000001_0, > Status : FAILED > 07/08/02 09:24:04 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:24:04 WARN mapred.JobClient: Error reading task > outputhadoop102.e-manation.com > 07/08/02 09:24:26 INFO mapred.JobClient: map 100% reduce 7% > 07/08/02 09:24:26 INFO mapred.JobClient: Task Id : task_0002_m_000001_0, > Status : FAILED > 07/08/02 09:24:26 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:24:26 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:24:26 INFO mapred.JobClient: Task Id : task_0002_r_000000_0, > Status : FAILED > 07/08/02 09:24:26 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:24:26 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:24:33 INFO mapred.JobClient: map 100% reduce 33% > 07/08/02 09:24:37 INFO mapred.JobClient: map 100% reduce 66% > 07/08/02 09:25:11 INFO mapred.JobClient: Task Id : task_0002_r_000001_1, > Status : FAILED > 07/08/02 09:25:11 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:25:11 WARN mapred.JobClient: Error reading task > outputhadoop91.e-manation.com > 07/08/02 09:25:17 INFO mapred.JobClient: map 100% reduce 100% > 07/08/02 09:25:18 INFO mapred.JobClient: Job complete: job_0002 > 07/08/02 09:25:18 INFO mapred.JobClient: Counters: 7 > 07/08/02 09:25:18 INFO mapred.JobClient: Map-Reduce Framework > 07/08/02 09:25:18 INFO mapred.JobClient: Map input records=1 > 07/08/02 09:25:18 INFO mapred.JobClient: Map output records=1 > 07/08/02 09:25:18 INFO mapred.JobClient: Map input bytes=23 > 07/08/02 09:25:18 INFO mapred.JobClient: Map output bytes=15 > 07/08/02 09:25:18 INFO mapred.JobClient: Reduce input groups=1 > 07/08/02 09:25:18 INFO mapred.JobClient: Reduce input records=1 > 07/08/02 09:25:18 INFO mapred.JobClient: Reduce output records=1 > > Every time the WARN and FAIL appears is when I kill and restart a > tasktracker. > What's really disheartening is that while the reduce job goes further > and further, some tasktrackers become like "lazy", staying passive like > unrecognized by the jobtracker or I don't know... so that the mess ends > as if there were only 1 tasktracker slave working. > Actually, as soon as 2 tasktrackers or more are working at the same > time, the job sticks... > Anyone, any idea ?? > > Sam > > > > > > Konstantin Shvachko a écrit : > > It seems like you are hitting the problem discussed in HADOOP-1374. > > We still don't know what causes it. It would be very good if you share > > your experience in this issue. > > Do you use hadoop scripts to run your tests? Which platform are you > > running them on? > > > > In my experience sometimes it helps if you change the order in which > > the slaves start. > > If that does not help try to kill the task-tracker that is stuck and > > restart it later. > > The reduce task will be rescheduled to the other task-tracker and will > > succeed. > > > > Thanks, > > Konstantin > > > > Samuel LEMOINE wrote: > > > >> Hi everyone ! > >> > >> I'm still trying to understand the way hadoop works, and the > >> possibilities offered in parallelizing java applications with haddop > >> (especially lucene-based ones). > >> For the moment, I've focused my efforts on the examples given (Grep > >> and WordCount). > >> I've managed to make both of them work with a hadoop running as > >> "file:///" namenode. > >> When I try it on dfs monde (namenode configured to my local ip), Grep > >> still works, but WordCount doesn't anymore. > >> I manage to make Grep work with 1 computer for namenode and another > >> for 1 datanode (obviously, WordCount still doesn't work). > >> > >> But as soon as I try to built some kind of mini "real" cluster, with > >> a namenode and n datanodes, with n=2, the map/reduce blocks. It > >> doesn't produce any error, and the logs files don't show anything > >> strange, the execution just freezes a little time after the beginning > >> of the reduce task. I've noticed that the reduce task begins before > >> the end of the map task when 2 slaves are availables, which is not > >> the case with only 1 slave. I know that it's the expected behaviour, > >> but I suspect it to be the cause of the freeze. > >> The execution is doesn't follow exactly the same way each time, > >> sometimes it blocks on 91%map/31%reduce, other times > >> 89%map/15%reduce, but it's always at the beginning of the reduce > >> task. I've tryed to let it run for a whole night in case it was just > >> veeeeryyyy long, but it didn't go any further. > >> The 2 slaves are quite identical, and both of them work when alone as > >> slave. > >> > >> I'm blocked at this point for about 5 days, hanging around with no > >> track to explore anymore. Any help will be greatly appreciated, any > >> clue or so... if someone knows a good way for monitoring the distant > >> java-tasks launched on the distant jobtrackers/tasktrackers, it's > >> also welcome :) > >> > >> Oh, a little precision that shouldn't change much things: the 2 > >> slaves are virtual machines, with 256MB of RAM each. All access to > >> them are through ssh (which is quite boring when you change the > >> config files a lot of times to try the different possibilities :D ) > >> > >> > >> Samuel > >> > >> > >> PS: I promise to share my hadoop experience as soon as it takes a > >> consistent shape. I anticipate rediging a tutorial for the company > >> where i'm doing my internship, and then translate it in english for > >> the wiki. > >> PS2: here follow the console messages for a few of my attempts; > >> > >> console messages while running Grep on a 1master-2slaves dfs > >> architecture: > >> > >> /opt/java/bin/java -Didea.launcher.port=7540 > >> -Didea.launcher.bin.path=/opt/idea-6180/bin -Dfile.encoding=UTF-8 > >> -classpath > >> > /opt/jdk1.5.0_12/jre/lib/charsets.jar:/opt/jdk1.5.0_12/jre/lib/jce.jar:/opt/jdk1.5.0_12/jre/lib/jsse.jar:/opt/jdk1.5.0_12/jre/lib/plugin.jar:/opt/jdk1.5.0_12/jre/lib/deploy.jar:/opt/jdk1.5.0_12/jre/lib/javaws.jar:/opt/jdk1.5.0_12/jre/lib/rt.jar:/opt/jdk1.5.0_12/jre/lib/ext/localedata.jar:/opt/jdk1.5.0_12/jre/lib/ext/dnsns.jar:/opt/jdk1.5.0_12/jre/lib/ext/sunpkcs11.jar:/opt/jdk1.5.0_12/jre/lib/ext/sunjce_provider.jar:/home/samuel/IdeaProjects/hadoopTest/classes/test/hadoopTest:/home/samuel/IdeaProjects/hadoopTest/classes/production/hadoopTest:/home/samuel/commons- > logging-1.1/commons- > logging-1.1.jar:/home/samuel/IdeaProjects/hadoopTest/lib/log4j/log4j-1.2.14.jar:/home/samuel/IdeaProjects/hadoopTest/lib/commons-cli-1.1.jar:/home/samuel/IdeaProjects/hadoopTest/lib/http/commons-codec-1.3.jar:/home/samuel/IdeaProjects/hadoopTest/lib/http/commons-logging-1.0.4.jar:/home/samuel/IdeaProjects/hadoopTest/lib/http/httpclient-4.0-alpha1.jar:/home/samuel/IdeaProjects/hadoopTest/lib/http/ht > >> > tpcore-4.0-alpha5.jar:/home/samuel/IdeaProjects/hadoopTest/lib/commons-httpclient-3.0.1/commons-httpclient-3.0.1.jar:/home/samuel/IdeaProjects/hadoopTest/hadoop/conf:/home/samuel/hadoop-0.13.1/hadoop-0.13.1-core.jar:/opt/idea-6180/lib/idea_rt.jar > >> com.intellij.rt.execution.application.AppMain > >> com.lingway.hadoopScratchPad.Grep /user/hadoop/documents > >> /user/hadoop/results blabla > >> 07/08/01 11:14:01 INFO mapred.FileInputFormat: Total input paths to > >> process : 19 > >> 07/08/01 11:14:03 INFO mapred.JobClient: Running job: job_0001 > >> 07/08/01 11:14:04 INFO mapred.JobClient: map 0% reduce 0% > >> 07/08/01 11:14:14 INFO mapred.JobClient: map 10% reduce 0% > >> 07/08/01 11:14:15 INFO mapred.JobClient: map 21% reduce 0% > >> 07/08/01 11:14:16 INFO mapred.JobClient: map 26% reduce 0% > >> 07/08/01 11:14:17 INFO mapred.JobClient: map 31% reduce 0% > >> 07/08/01 11:14:18 INFO mapred.JobClient: map 42% reduce 0% > >> 07/08/01 11:14:19 INFO mapred.JobClient: map 47% reduce 0% > >> 07/08/01 11:14:21 INFO mapred.JobClient: map 57% reduce 0% > >> 07/08/01 11:14:22 INFO mapred.JobClient: map 63% reduce 0% > >> 07/08/01 11:14:23 INFO mapred.JobClient: map 73% reduce 0% > >> 07/08/01 11:14:24 INFO mapred.JobClient: map 78% reduce 0% > >> 07/08/01 11:14:25 INFO mapred.JobClient: map 84% reduce 0% > >> 07/08/01 11:14:26 INFO mapred.JobClient: map 89% reduce 0% > >> 07/08/01 11:14:34 INFO mapred.JobClient: map 89% reduce 15% > >> > >> //and then doesn't go any further > >> > >> > >> > >> > >> > ******************************************************************************************** > >> > >> > >> console messages while running WordCount on a single-node dfs > >> architecture: > >> > >> /opt/java/bin/java -Didea.launcher.port=7537 > >> -Didea.launcher.bin.path=/opt/idea-6180/bin -Dfile.encoding=UTF-8 > >> -classpath > >> > /opt/jdk1.5.0_12/jre/lib/charsets.jar:/opt/jdk1.5.0_12/jre/lib/jce.jar:/opt/jdk1.5.0_12/jre/lib/jsse.jar:/opt/jdk1.5.0_12/jre/lib/plugin.jar:/opt/jdk1.5.0_12/jre/lib/deploy.jar:/opt/jdk1.5.0_12/jre/lib/javaws.jar:/opt/jdk1.5.0_12/jre/lib/rt.jar:/opt/jdk1.5.0_12/jre/lib/ext/localedata.jar:/opt/jdk1.5.0_12/jre/lib/ext/dnsns.jar:/opt/jdk1.5.0_12/jre/lib/ext/sunpkcs11.jar:/opt/jdk1.5.0_12/jre/lib/ext/sunjce_provider.jar:/home/samuel/IdeaProjects/hadoopTest/classes/test/hadoopTest:/home/samuel/IdeaProjects/hadoopTest/classes/production/hadoopTest:/home/samuel/commons- > logging-1.1/commons- > logging-1.1.jar:/home/samuel/IdeaProjects/hadoopTest/lib/log4j/log4j-1.2.14.jar:/home/samuel/IdeaProjects/hadoopTest/lib/commons-cli-1.1.jar:/home/samuel/IdeaProjects/hadoopTest/lib/http/commons-codec-1.3.jar:/home/samuel/IdeaProjects/hadoopTest/lib/http/commons-logging-1.0.4.jar:/home/samuel/IdeaProjects/hadoopTest/lib/http/httpclient-4.0-alpha1.jar:/home/samuel/IdeaProjects/hadoopTest/lib/http/ht > >> > tpcore-4.0-alpha5.jar:/home/samuel/IdeaProjects/hadoopTest/lib/commons-httpclient-3.0.1/commons-httpclient-3.0.1.jar:/home/samuel/IdeaProjects/hadoopTest/hadoop/conf:/home/samuel/hadoop-0.13.1/hadoop-0.13.1-core.jar:/opt/idea-6180/lib/idea_rt.jar > >> com.intellij.rt.execution.application.AppMain > >> com.lingway.hadoopScratchPad.WordCount /user/hadoop/documents > >> /user/hadoop/resultats > >> 07/08/01 10:47:41 INFO mapred.FileInputFormat: Total input paths to > >> process : 19 > >> 07/08/01 10:47:42 INFO mapred.JobClient: Running job: job_0004 > >> 07/08/01 10:47:43 INFO mapred.JobClient: map 0% reduce 0% > >> 07/08/01 10:47:48 INFO mapred.JobClient: Task Id : > >> task_0004_m_000000_0, Status : FAILED > >> 07/08/01 10:47:52 INFO mapred.JobClient: Task Id : > >> task_0004_m_000001_0, Status : FAILED > >> 07/08/01 10:47:57 INFO mapred.JobClient: Task Id : > >> task_0004_m_000002_0, Status : FAILED > >> 07/08/01 10:48:01 INFO mapred.JobClient: Task Id : > >> task_0004_m_000003_0, Status : FAILED > >> 07/08/01 10:48:05 INFO mapred.JobClient: Task Id : > >> task_0004_m_000004_0, Status : FAILED > >> 07/08/01 10:48:10 INFO mapred.JobClient: Task Id : > >> task_0004_m_000005_0, Status : FAILED > >> 07/08/01 10:48:14 INFO mapred.JobClient: Task Id : > >> task_0004_m_000006_0, Status : FAILED > >> 07/08/01 10:48:18 INFO mapred.JobClient: Task Id : > >> task_0004_m_000007_0, Status : FAILED > >> 07/08/01 10:48:24 INFO mapred.JobClient: Task Id : > >> task_0004_m_000008_0, Status : FAILED > >> 07/08/01 10:48:28 INFO mapred.JobClient: Task Id : > >> task_0004_m_000009_0, Status : FAILED > >> 07/08/01 10:48:32 INFO mapred.JobClient: Task Id : > >> task_0004_m_000010_0, Status : FAILED > >> 07/08/01 10:48:36 INFO mapred.JobClient: Task Id : > >> task_0004_m_000011_0, Status : FAILED > >> 07/08/01 10:48:41 INFO mapred.JobClient: Task Id : > >> task_0004_m_000012_0, Status : FAILED > >> 07/08/01 10:48:45 INFO mapred.JobClient: Task Id : > >> task_0004_m_000013_0, Status : FAILED > >> 07/08/01 10:48:49 INFO mapred.JobClient: Task Id : > >> task_0004_m_000014_0, Status : FAILED > >> 07/08/01 10:48:54 INFO mapred.JobClient: Task Id : > >> task_0004_m_000015_0, Status : FAILED > >> 07/08/01 10:48:58 INFO mapred.JobClient: Task Id : > >> task_0004_m_000016_0, Status : FAILED > >> 07/08/01 10:49:02 INFO mapred.JobClient: Task Id : > >> task_0004_m_000017_0, Status : FAILED > >> 07/08/01 10:49:06 INFO mapred.JobClient: Task Id : > >> task_0004_m_000018_0, Status : FAILED > >> 07/08/01 10:49:11 INFO mapred.JobClient: Task Id : > >> task_0004_m_000000_1, Status : FAILED > >> 07/08/01 10:49:15 INFO mapred.JobClient: Task Id : > >> task_0004_m_000000_2, Status : FAILED > >> 07/08/01 10:49:19 INFO mapred.JobClient: map 100% reduce 100% > >> Exception in thread "main" java.io.IOException: Job failed! > >> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:604) > >> at com.lingway.hadoopScratchPad.WordCount.main(WordCount.java:145) > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> at > >> sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:39) > >> > >> at > >> sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:25) > >> > >> at java.lang.reflect.Method.invoke(Method.java:585) > >> at > >> com.intellij.rt.execution.application.AppMain.main(AppMain.java:90) > >> > >> Process finished with exit code 1 > >> > >> > >> > >> > >> > >> > >> > >> > >> > ******************************************************************************************** > >> > >> > >> > >> console messages while running WordCount on a 1master-1slave dfs > >> architecture: > >> > >> /opt/java/bin/java -Didea.launcher.port=7539 > >> -Didea.launcher.bin.path=/opt/idea-6180/bin -Dfile.encoding=UTF-8 > >> -classpath > >> > /opt/jdk1.5.0_12/jre/lib/charsets.jar:/opt/jdk1.5.0_12/jre/lib/jce.jar:/opt/jdk1.5.0_12/jre/lib/jsse.jar:/opt/jdk1.5.0_12/jre/lib/plugin.jar:/opt/jdk1.5.0_12/jre/lib/deploy.jar:/opt/jdk1.5.0_12/jre/lib/javaws.jar:/opt/jdk1.5.0_12/jre/lib/rt.jar:/opt/jdk1.5.0_12/jre/lib/ext/localedata.jar:/opt/jdk1.5.0_12/jre/lib/ext/dnsns.jar:/opt/jdk1.5.0_12/jre/lib/ext/sunpkcs11.jar:/opt/jdk1.5.0_12/jre/lib/ext/sunjce_provider.jar:/home/samuel/IdeaProjects/hadoopTest/classes/test/hadoopTest:/home/samuel/IdeaProjects/hadoopTest/classes/production/hadoopTest:/home/samuel/commons- > logging-1.1/commons- > logging-1.1.jar:/home/samuel/IdeaProjects/hadoopTest/lib/log4j/log4j-1.2.14.jar:/home/samuel/IdeaProjects/hadoopTest/lib/commons-cli-1.1.jar:/home/samuel/IdeaProjects/hadoopTest/lib/http/commons-codec-1.3.jar:/home/samuel/IdeaProjects/hadoopTest/lib/http/commons-logging-1.0.4.jar:/home/samuel/IdeaProjects/hadoopTest/lib/http/httpclient-4.0-alpha1.jar:/home/samuel/IdeaProjects/hadoopTest/lib/http/ht > >> > tpcore-4.0-alpha5.jar:/home/samuel/IdeaProjects/hadoopTest/lib/commons-httpclient-3.0.1/commons-httpclient-3.0.1.jar:/home/samuel/IdeaProjects/hadoopTest/hadoop/conf:/home/samuel/hadoop-0.13.1/hadoop-0.13.1-core.jar:/opt/idea-6180/lib/idea_rt.jar > >> com.intellij.rt.execution.application.AppMain > >> com.lingway.hadoopScratchPad.WordCount /user/hadoop/documents > >> /user/hadoop/resultats > >> 07/08/01 11:08:24 INFO mapred.FileInputFormat: Total input paths to > >> process : 19 > >> 07/08/01 11:08:26 INFO mapred.JobClient: Running job: job_0001 > >> 07/08/01 11:08:27 INFO mapred.JobClient: map 0% reduce 0% > >> 07/08/01 11:08:37 INFO mapred.JobClient: Task Id : > >> task_0001_m_000002_0, Status : FAILED > >> 07/08/01 11:08:37 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:37 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:37 INFO mapred.JobClient: Task Id : > >> task_0001_m_000000_0, Status : FAILED > >> 07/08/01 11:08:37 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:37 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:41 INFO mapred.JobClient: Task Id : > >> task_0001_m_000005_0, Status : FAILED > >> 07/08/01 11:08:41 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:41 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:42 INFO mapred.JobClient: Task Id : > >> task_0001_m_000004_0, Status : FAILED > >> 07/08/01 11:08:42 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:42 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:46 INFO mapred.JobClient: Task Id : > >> task_0001_m_000006_0, Status : FAILED > >> 07/08/01 11:08:46 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:46 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:47 INFO mapred.JobClient: Task Id : > >> task_0001_m_000001_0, Status : FAILED > >> 07/08/01 11:08:47 INFO mapred.JobClient: Task Id : > >> task_0001_m_000007_0, Status : FAILED > >> 07/08/01 11:08:47 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:48 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:48 INFO mapred.JobClient: Task Id : > >> task_0001_m_000003_0, Status : FAILED > >> 07/08/01 11:08:52 INFO mapred.JobClient: Task Id : > >> task_0001_m_000008_0, Status : FAILED > >> 07/08/01 11:08:52 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:52 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:52 INFO mapred.JobClient: Task Id : > >> task_0001_m_000009_0, Status : FAILED > >> 07/08/01 11:08:52 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:52 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:57 INFO mapred.JobClient: Task Id : > >> task_0001_m_000010_0, Status : FAILED > >> 07/08/01 11:08:57 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:57 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:57 INFO mapred.JobClient: Task Id : > >> task_0001_r_000000_0, Status : FAILED > >> 07/08/01 11:08:57 INFO mapred.JobClient: Task Id : > >> task_0001_m_000011_0, Status : FAILED > >> 07/08/01 11:08:57 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:08:57 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:02 INFO mapred.JobClient: Task Id : > >> task_0001_m_000012_0, Status : FAILED > >> 07/08/01 11:09:02 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:02 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:02 INFO mapred.JobClient: Task Id : > >> task_0001_m_000013_0, Status : FAILED > >> 07/08/01 11:09:02 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:02 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:02 INFO mapred.JobClient: Task Id : > >> task_0001_m_000002_1, Status : FAILED > >> 07/08/01 11:09:02 INFO mapred.JobClient: Task Id : > >> task_0001_m_000000_1, Status : FAILED > >> 07/08/01 11:09:07 INFO mapred.JobClient: Task Id : > >> task_0001_m_000014_0, Status : FAILED > >> 07/08/01 11:09:07 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:07 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:07 INFO mapred.JobClient: Task Id : > >> task_0001_m_000015_0, Status : FAILED > >> 07/08/01 11:09:07 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:07 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:11 INFO mapred.JobClient: Task Id : > >> task_0001_m_000016_0, Status : FAILED > >> 07/08/01 11:09:11 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:11 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:12 INFO mapred.JobClient: Task Id : > >> task_0001_m_000017_0, Status : FAILED > >> 07/08/01 11:09:12 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:12 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:16 INFO mapred.JobClient: Task Id : > >> task_0001_m_000018_0, Status : FAILED > >> 07/08/01 11:09:16 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:16 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:16 INFO mapred.JobClient: Task Id : > >> task_0001_m_000001_1, Status : FAILED > >> 07/08/01 11:09:16 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:16 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:17 INFO mapred.JobClient: Task Id : > >> task_0001_m_000004_1, Status : FAILED > >> 07/08/01 11:09:18 INFO mapred.JobClient: Task Id : > >> task_0001_m_000005_1, Status : FAILED > >> 07/08/01 11:09:21 INFO mapred.JobClient: Task Id : > >> task_0001_m_000000_2, Status : FAILED > >> 07/08/01 11:09:21 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:21 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:21 INFO mapred.JobClient: Task Id : > >> task_0001_m_000003_1, Status : FAILED > >> 07/08/01 11:09:21 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:21 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:27 INFO mapred.JobClient: map 100% reduce 100% > >> 07/08/01 11:09:27 INFO mapred.JobClient: Task Id : > >> task_0001_m_000001_2, Status : FAILED > >> 07/08/01 11:09:27 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> 07/08/01 11:09:27 WARN mapred.JobClient: Error reading task > >> outputubuntu704.e-manation.com > >> Exception in thread "main" java.io.IOException: Job failed! > >> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:604) > >> at com.lingway.hadoopScratchPad.WordCount.main(WordCount.java:145) > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> at > >> sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:39) > >> > >> at > >> sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:25) > >> > >> at java.lang.reflect.Method.invoke(Method.java:585) > >> at > >> com.intellij.rt.execution.application.AppMain.main(AppMain.java:90) > >> > >> Process finished with exit code 1 > >> > >> > >> > > > > > > > > >
