Emit (key, value) twice from mapper by modifying key as key' = (key, partId) and record becomes (key', value) >From custom partitioner, send record to reducer based on partId. Ignore partId >field in reducer.
Regards, Ajay Srivastava On 13-Mar-2013, at 2:29 PM, Vikas Jadhav wrote: Hi I am specifying requirement again with example. I have use case where i need to shufffle same (key,value) pair to multiple reducers For Example we have pair (1,"ABC") and two reducers (reducer0 and reducer1) are there then by default this pair will go to reduce1 (cause (key % numOfReducer) = (1%2) ) how i should shuffle this pair to both reducer. Also I willing to change the code of hadoop framework if Necessory. Thank you On Wed, Mar 13, 2013 at 12:51 PM, feng lu <[email protected]<mailto:[email protected]>> wrote: Hi you can use Job#setNumReduceTasks(int tasks) method to set the number of reducer to output. On Wed, Mar 13, 2013 at 2:15 PM, Vikas Jadhav <[email protected]<mailto:[email protected]>> wrote: Hello, As by default Hadoop framework can shuffle (key,value) pair to only one reducer I have use case where i need to shufffle same (key,value) pair to multiple reducers Also I willing to change the code of hadoop framework if Necessory. Thank you -- Thanx and Regards Vikas Jadhav -- Don't Grow Old, Grow Up... :-) -- Thanx and Regards Vikas Jadhav
