Hi Sasha By default the number or reducers are set to be 1. If you want more you need to specify it as
hadoop jar myJar.jar myClass -D mapred.reduce.tasks=20 ... Regards Bejoy KS Sent from handheld, please excuse typos. -----Original Message----- From: jamal sasha <[email protected]> Date: Tue, 20 Nov 2012 14:38:54 To: <[email protected]> Reply-To: [email protected] Subject: number of reducers Hi, I wrote a simple map reduce job in hadoop streaming. I am wondering if I am doing something wrong .. While number of mappers are projected to be around 1700.. reducers.. just 1? It’s couple of TB’s worth of data. What can I do to address this. Basically mapper looks like this For line in sys.stdin: Print line Reducer For line in sys.stdin: New_line = process_line(line) Print new_line Thanks
