how to debug a fully-distributed wordcount performance? [11:01] <r|away> I have installed Hadoop on standalone, pseudo distributed, and fully distributed, [11:01] <r|away> then run the hadoop-0.16.4-examples.jar wordcount, [11:02] <r|away> I found the wordcount can run quickly on local and pseudo mode, [11:02] <r|away> but a little slower on 2 DataNode fully distributed, [11:03] <r|away> very slow on 4 nodes, [11:03] <r|away> why that happened? [11:04] <r|away> the more nodes, the slower it run? [11:04] <r|away> # time bin/hadoop jar hadoop-0.16.4-examples.jar wordcount input output [11:05] <r|away> a recent result is 12 minutes! the input/ directory has only one "Hello, World" file, [11:05] <r|away> who knows why?
-- Cheng
