Hello Vikas! Well you can duplicate your pair in mapper for each reducer, add reducer's mark to keys and write a partitioner that will partition records acording this marks.
I mean something like this: In mapper you produce (key_r1 value) (key_r2 value) instead (key value) But i cannot imagine why you need that. WBR, Onuchin Artem 2013/3/13 Viral Bajaria <[email protected]> > Do you want the pair to go to both reducers or do you want it to go to > only one but in a random fashion ? > > AFAIK, 1st is not possible. Someone on the list can correct if I am wrong. > 2nd is possible by just implementing your own partitioner which randomizes > where each key goes (not sure what you gain by that). > > > On Wed, Mar 13, 2013 at 1:59 AM, Vikas Jadhav <[email protected]>wrote: > >> >> Hi >> I am specifying requirement again with example. >> >> >> >> I have use case where i need to shufffle same (key,value) pair to >> multiple reducers >> >> >> For Example we have pair (1,"ABC") and two reducers (reducer0 and >> reducer1) are there then >> >> by default this pair will go to reduce1 (cause (key % numOfReducer) = >> (1%2) ) >> >> >> how i should shuffle this pair to both reducer. >> >> Also I willing to change the code of hadoop framework if Necessory. >> >> Thank you >> >> On Wed, Mar 13, 2013 at 12:51 PM, feng lu <[email protected]> wrote: >> >>> Hi >>> >>> you can use Job#setNumReduceTasks(int tasks) method to set the number of >>> reducer to output. >>> >>> >>> On Wed, Mar 13, 2013 at 2:15 PM, Vikas Jadhav >>> <[email protected]>wrote: >>> >>>> Hello, >>>> >>>> As by default Hadoop framework can shuffle (key,value) pair to only one >>>> reducer >>>> >>>> I have use case where i need to shufffle same (key,value) pair to >>>> multiple reducers >>>> >>>> Also I willing to change the code of hadoop framework if Necessory. >>>> >>>> >>>> Thank you >>>> >>>> -- >>>> * >>>> * >>>> * >>>> >>>> Thanx and Regards* >>>> * Vikas Jadhav* >>>> >>> >>> >>> >>> -- >>> Don't Grow Old, Grow Up... :-) >>> >> >> >> >> -- >> * >> * >> * >> >> Thanx and Regards* >> * Vikas Jadhav* >> > >
