Please help me, I can't figure out how to fix this problem. I have a cluster of virtual machines under VMWare (windows XP is original OS):
Ubuntu 8.10 Intel Pentium DUAL CPU E2180 @ 2 GHZ Memory 1024 MB I have a namenode and 8 more datanodes. I want to start teragen and terasort programs and do a benchmark analysis of a cluster running 1, 3 and all 8 datanodes. Datanodes have only 20GB configured HDFS capacity each, so it is a total of cca 150GB total. I have no problem generating the input data with 2 or 8 maps but problem comes out with terasort. When it comes to reduce phase, it generates a following error: 10/07/13 10:59:40 INFO mapred.JobClient: Task Id : attempt_201007131052_0002_r_000000_0, Status : FAILED Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out. As I understand I have to setup these parameters in mapred-site.xml to override default values: <property> <name>mapred.map.tasks</name> <value>?</value> </property> <property> <name>mapred.reduce.tasks</name> <value>?</value> </property> Does anyone know how to setup number of reducers so that it works :). Thank you...
