[
https://issues.apache.org/jira/browse/TEZ-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuhu Shukla updated TEZ-3605:
-----------------------------
Attachment: TEZ-3605.006.patch
Updated patch that moves incrementing the numRecordsPerPartition values to a
method. Currently I have replicated the idea from Unordered case to keep track
of records per partition, alternatively we could make it a bitset but it might
be helpful later to have an array of records per partition for any usage by
extension. Also added getOutputContext getter and modified the test to assert
for more conditions like the bitset for empty partitions being set correctly
etc.
> Detect and prune empty partitions for the Ordered case
> ------------------------------------------------------
>
> Key: TEZ-3605
> URL: https://issues.apache.org/jira/browse/TEZ-3605
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
> Attachments: TEZ-3605.001.patch, TEZ-3605.002.patch,
> TEZ-3605.003.patch, TEZ-3605.004.patch, TEZ-3605.005.patch, TEZ-3605.006.patch
>
>
> Analogous to the Unordered case we should not have empty partition
> entries/segments in the Ordered/DefaultSorter case. This will save writing
> unnecessary data.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
