[
https://issues.apache.org/jira/browse/TEZ-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16031088#comment-16031088
]
Kuhu Shukla commented on TEZ-3605:
----------------------------------
TestExceptionPropagation failure seems unrelated and locally irreproducible. I
will continue to investigate this. Other test failures are known and already
have JIRAs associated. [~jeagles], looking for some comments on the latest
patch. Thanks a lot!
> Detect and prune empty partitions for the Ordered case
> ------------------------------------------------------
>
> Key: TEZ-3605
> URL: https://issues.apache.org/jira/browse/TEZ-3605
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
> Attachments: TEZ-3605.001.patch, TEZ-3605.002.patch,
> TEZ-3605.003.patch, TEZ-3605.004.patch, TEZ-3605.005.patch,
> TEZ-3605.006.patch, TEZ-3605.007.patch, TEZ-3605.008.patch,
> TEZ-3605.009.patch, TEZ-3605.010.patch
>
>
> Analogous to the Unordered case we should not have empty partition
> entries/segments in the Ordered/DefaultSorter case. This will save writing
> unnecessary data.
> Additionally, with tez_shuffle feature (TEZ-3334), in a heavily auto reduced
> job, this change would allow not fetching empty partitions and then throwing
> them away.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
