[
https://issues.apache.org/jira/browse/TEZ-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716570#comment-15716570
]
Zhiyuan Yang commented on TEZ-3552:
-----------------------------------
[~mingma] Are you going to work on this? If not, I'll take it over. Do you
think it's necessary to add a another knob for whether to shuffle splits? I
prefer simply to do randomization if sorting is turned off.
> Shuffle split array when size-based sorting is turned off
> ---------------------------------------------------------
>
> Key: TEZ-3552
> URL: https://issues.apache.org/jira/browse/TEZ-3552
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Ming Ma
>
> TEZ-3430 adds the functionality to skip size-based split sorting to help with
> job runtime. During further testing, the original split array for certain
> inputs before sorting aren't randomly distributed in size. So when the spit
> sorting is turned off, we should shuffle the split instead of doing nothing.
> That will make the size distribution more even.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
