On 01/18/2011 03:09 PM, Mapred Learn wrote:
hi,
I have a basic question. How does partitioning work ?
Following is a scenario I created to put up my question.
i) A parttition function is defined as partitioning map-output based on
aphabetical sorting of the key i.e. a partition for keys starting with 'a',
partition for keys starting with 'b'... partition for keys starting with
'z'. So, it means each map may have atmost 26 partitions ?
ii) What input will Reducer get ? Reducer will get first partition
(partition starting with 'a') of all the maps as it's input ? Does it mean
we will need 26 reduce tasks ?
Any inputs/documents/examples on this are appreciated. I am bit confused by
this.
Thanks in advance
You should probably read the Yahoo tutorial to brush up on the topic
before asking on the list.
http://developer.yahoo.com/hadoop/tutorial/module5.html#partitioning
If you still don't understand after that, and you post a specific
question (i.e., not "how does partitioning work") I'm sure someone will
be able to answer.
DR
