[
https://issues.apache.org/jira/browse/TEZ-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14104721#comment-14104721
]
Siddharth Seth commented on TEZ-1448:
-------------------------------------
{code} static final double MAX_ADDITIONAL_RESERVATION_FRACTION_PER_IO = 0.03d;
{code}
This value seems far too low. Inputs typically have an overhead beyond the
standard buffers they allocate. Should this be closer to 0.1 ?
{code}static String[] generateWeightStrings(int broadcastIn, int sortedOut, int
scatterGatherShuffleIn,{code}
Please change the unit test to make use of this method as well; it has the same
code otherwise.
RequestType for UNSORTED_OUTPUT (without partitioning is missing). Can you
please add that as part of this patch.
Otherwise, looks good.
> Make WeightedScalingMemoryDistributor as the default memory distributor
> -----------------------------------------------------------------------
>
> Key: TEZ-1448
> URL: https://issues.apache.org/jira/browse/TEZ-1448
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-1448.1.patch
>
>
> Currently Tez uses ScalingAllocator; WeightedScalingMemoryDistributor
> provides better control over ScalingAllocator.
> In certain DAG executions, response time improvements of over 6% have been
> observed at 10 TB scale and ~26% runtime improvement in memory intensive
> reducers.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
