[ 
https://issues.apache.org/jira/browse/HADOOP-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12535529
 ] 

Runping Qi commented on HADOOP-1965:
------------------------------------


It seems clear that threaded spill performed much better than sequence spill.

One thing surprising is that the spill times got worse as sort.io.mb increaseed.

This sounds counterintuitive. Any insights/explanations?


> Handle map output buffers better
> --------------------------------
>
>                 Key: HADOOP-1965
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1965
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Devaraj Das
>            Assignee: Amar Kamat
>         Attachments: 1965_single_proc_150mb_gziped.pdf
>
>
> Today, the map task stops calling the map method while sort/spill is using 
> the (single instance of) map output buffer. One improvement that can be done 
> to improve performance of the map task is to have another buffer for writing 
> the map outputs to, while sort/spill is using the first buffer.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to