[
https://issues.apache.org/jira/browse/TEZ-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuhu Shukla updated TEZ-2950:
-----------------------------
Assignee: Kuhu Shukla
Assigning this to myself. Will post a proposal asap.
> Poor performance of UnorderedPartitionedKVWriter
> ------------------------------------------------
>
> Key: TEZ-2950
> URL: https://issues.apache.org/jira/browse/TEZ-2950
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Kuhu Shukla
>
> Came across a job which was taking a long time in
> UnorderedPartitionedKVWriter.mergeAll. It was decompressing and reading data
> from spill files (8500 spills) and then writing the final compressed merge
> file. Why do we need spill files for UnorderedPartitionedKVWriter? Why not
> just buffer and keep directly writing to the final file which will save a lot
> of time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
