[
https://issues.apache.org/jira/browse/TEZ-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated TEZ-1448:
----------------------------------
Attachment: TEZ-1448.3.patch
Thanks @sseth. Made equal distribution for SortedMergedInput and SortedOutput
in the attachment, which should work for most of the jobs.
For some jobs, it would be beneficial to tweak the settings to allow more
memory for SortedOutput (to avoid spilling), which can be tweaked by users.
> Make WeightedScalingMemoryDistributor as the default memory distributor
> -----------------------------------------------------------------------
>
> Key: TEZ-1448
> URL: https://issues.apache.org/jira/browse/TEZ-1448
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-1448.1.patch, TEZ-1448.2.patch, TEZ-1448.3.patch
>
>
> Currently Tez uses ScalingAllocator; WeightedScalingMemoryDistributor
> provides better control over ScalingAllocator.
> In certain DAG executions, response time improvements of over 6% have been
> observed at 10 TB scale and ~26% runtime improvement in memory intensive
> reducers.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
