[
https://issues.apache.org/jira/browse/TEZ-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064752#comment-16064752
]
Kuhu Shukla commented on TEZ-3605:
----------------------------------
Request for comments/review on the latest patch [~sseth], [~jeagles]. Thanks a
lot!
> Detect and prune empty partitions for the Ordered case
> ------------------------------------------------------
>
> Key: TEZ-3605
> URL: https://issues.apache.org/jira/browse/TEZ-3605
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
> Attachments: TEZ-3605.001.patch, TEZ-3605.002.patch,
> TEZ-3605.003.patch, TEZ-3605.004.patch, TEZ-3605.005.patch,
> TEZ-3605.006.patch, TEZ-3605.007.patch, TEZ-3605.008.patch,
> TEZ-3605.009.patch, TEZ-3605.010.patch, TEZ-3605.011.patch, TEZ-3605.012.patch
>
>
> Analogous to the Unordered case we should not have empty partition
> entries/segments in the Ordered/DefaultSorter case. This will save writing
> unnecessary data.
> Additionally, with tez_shuffle feature (TEZ-3334), in a heavily auto reduced
> job, this change would allow not fetching empty partitions and then throwing
> them away.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
