[
https://issues.apache.org/jira/browse/TEZ-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949724#comment-13949724
]
Siddharth Seth commented on TEZ-932:
------------------------------------
bq. there should be a big enough paragraph at the start explaining what this
class does and how it does it.
Added
bq. maybe slightly shorter properties if possible?
Shortened one of them to change max-total-limit to max. Any suggestions ? I'd
like it to be understandable at the same time.
Renamed to WeightedScalingMemoryDistributor
bq. wasnt there recent code that would start throwing an error if received 0
for available memory ?
Not sure how this is related. If users do configure something as 0, they'll see
an error.
bq. why is this not an error where the task is failed due to misconfiguration?
Changed to an error
bq. this does not seem right. What is the format? Seems to have 5 numbers which
are set without any clue as to what they map to.
The name is included in the conf. Have added an example to the code.
BROADCAST_INPUT:2,SORT_BUFFER:3,SHUFFLE_SORT_MERGE:6,PROCESSOR:1,OTHER:1
> Add a type aware Initial memory distributor
> -------------------------------------------
>
> Key: TEZ-932
> URL: https://issues.apache.org/jira/browse/TEZ-932
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Attachments: TEZ-932.1.txt, TEZ-932.2.txt, TEZ-932.3.txt
>
>
> The current memory scaler has not idea about the requirements of the
> underlying components (broadcast, shfufle etc). This jira is to add a scaler
> which can take the types into consideration.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
