Please try Hadoop 0.13.0.
The multiple random writers case now completes in 278sec (from 294sec). This diff is not big. In comparison, the sorter improvement is more impressive. For the sorter, now the map phase completes in a tighter range bound 55sec to 79sec (instead of 59sec to 139sec). This speedup (from scheduling?) the overall running time significantly (890sec vs 1345sec). The performance of reduce phase is similar (both shuffle and sort cases). Let me summarize my remaining questions in the next email. Looks like my original email is way too long to get specific answers. thanks bwolen
