[
https://issues.apache.org/jira/browse/TEZ-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093397#comment-15093397
]
Bikas Saha commented on TEZ-2978:
---------------------------------
Code changes lgtm.
This is effectively turning off rack or above grouping and essentially forcing
only node local groups even if they are small. So the config name should
reflect that to be more clear. Allow small groups early is unclear and leaks
impl logic. Something on the lines of - only node local - or - node locality
overrides size - may be more appropriate.
> Add an option to allow small splits early on in TezSplitGrouper
> ---------------------------------------------------------------
>
> Key: TEZ-2978
> URL: https://issues.apache.org/jira/browse/TEZ-2978
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Attachments: TEZ-2978.1.txt
>
>
> In certain cases, allowing the last few splits to be 'small' - smaller than
> the requested size is beneficial, and can end up creating fewer splits. Also
> this results in all splits having some form of locality information.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
