Re: Word Count sample

Wed, 23 May 2007 23:02:36 -0700 

Hi,
It seems that you are running the job in LocalJobRunner. Which means that the job runs only in your machine, not in the cluster. The job will have only one mapper in local mode, in which case it may take a while to process 240 MB of data. I can say that your configuration is not set properly. Please check your configuration(especially jobtracker) and retry. 


Phantom wrote:

I ran the sample against two files one of size 1.5 MB and the other 
240MB.
The bigger file just craps out. The map jobs do not seem to complete. 
Here

is the output from my run. Could someone please explain to me how I could
make sense of this output. Also how do I know on which machines my

map/reduce tasks are executing ? How do I get to the web interface to 
look

at statistics ?


java org.apache.hadoop.examples.WordCount -m 2 -r 3  
/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat /tmp/out-dir

07/05/23 15:24:32 INFO mapred.InputFormatBase: Total input paths to 
process

: 1
07/05/23 15:24:32 INFO mapred.JobClient: Running job: job_izgn5v
07/05/23 15:24:33 INFO mapred.JobClient:  map 0% reduce 0%

07/05/23 15:24:33 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:34 INFO mapred.JobClient:  map 4% reduce 0%

07/05/23 15:24:34 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:35 INFO mapred.JobClient:  map 12% reduce 0%

07/05/23 15:24:35 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:36 INFO mapred.JobClient:  map 19% reduce 0%

07/05/23 15:24:36 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:37 INFO mapred.JobClient:  map 26% reduce 0%

07/05/23 15:24:37 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:38 INFO mapred.JobClient:  map 33% reduce 0%

07/05/23 15:24:38 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:39 INFO mapred.JobClient:  map 41% reduce 0%

07/05/23 15:24:40 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:40 INFO mapred.JobClient:  map 47% reduce 0%

07/05/23 15:24:41 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:41 INFO mapred.JobClient:  map 55% reduce 0%

07/05/23 15:24:42 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:42 INFO mapred.JobClient:  map 63% reduce 0%

07/05/23 15:24:43 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:43 INFO mapred.JobClient:  map 69% reduce 0%

07/05/23 15:24:44 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:44 INFO mapred.JobClient:  map 77% reduce 0%

07/05/23 15:24:45 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:45 INFO mapred.JobClient:  map 83% reduce 0%

07/05/23 15:24:46 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:46 INFO mapred.JobClient:  map 90% reduce 0%

07/05/23 15:24:47 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432
07/05/23 15:24:47 INFO mapred.JobClient:  map 92% reduce 0%

07/05/23 15:24:48 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432

07/05/23 15:24:49 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432

07/05/23 15:24:50 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432

07/05/23 15:24:51 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432

07/05/23 15:24:52 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432

07/05/23 15:24:53 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432

07/05/23 15:24:54 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432

07/05/23 15:24:55 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432

07/05/23 15:24:56 INFO mapred.LocalJobRunner: 
file:/home/alakshman/hadoop-

0.12.3/in-dir/test2.dat:0+33554432

























					Reply via email to