[
https://issues.apache.org/jira/browse/TEZ-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994129#comment-15994129
]
Siddharth Seth commented on TEZ-3698:
-------------------------------------
In the close() method - !pipelinedShuffle, !finalMergeEnabled - if the
finalSpill() generates data, I don't think an event for this will be generated.
sendPipelinedEventForSpill - Please rename since this now handles
pipelinedShuffle as well as finalMergeEnabled==false. (It also doesn't always
send events)
Otherwise, looks good to me.
> UnorderedKV writer should be able to honor
> tez.runtime.enable.final-merge.in.output without pipelinedshuffle
> ------------------------------------------------------------------------------------------------------------
>
> Key: TEZ-3698
> URL: https://issues.apache.org/jira/browse/TEZ-3698
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-3698.1.patch
>
>
> Final merge can be disabled with "tez.runtime.enable.final-merge.in.output"
> setting. Currently this works with UnorderedKV writer only with pipelined
> shuffle. It should be able to honor this parameter, without pipelined shuffle
> as well to avoid final merge.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
