Hello everyone, I know when map func generates intermediate output, reduce func will pull data directly from all maps' local disk. Although we can use combiner func to minimize the amount of data, when we have many mappers, say 10,000, that will be a crazy IO headache. And that dosen't seem right.
Can anyone highlighten me on this? Regards, Elton -- View this message in context: http://old.nabble.com/What-it%27s-gonna-happend-when-it-comes-to-large-number-of-maps--tp28253798p28253798.html Sent from the Hadoop core-dev mailing list archive at Nabble.com.
