[
https://issues.apache.org/jira/browse/TEZ-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987986#comment-15987986
]
Harish Jaiprakash commented on TEZ-3701:
----------------------------------------
Thanks [~jlowe]. I found few test cases failing intermittently with
FileNotFound exceptions in finalSpill, when running tests for
UnorderedPartionedKVWriter with TEZ-3673 patch. I went through the code and
found no wait on the Future/executor service and I thought it was that. I
missed the pendingSpillCount wait. It might be something I've introduced as
part of TEZ-3673. I'll try to reproduce the test case failure and update this.
> UnorderedPartitionedKVWriter does not wait for submitted tasks to finish
> before finalMerge.
> -------------------------------------------------------------------------------------------
>
> Key: TEZ-3701
> URL: https://issues.apache.org/jira/browse/TEZ-3701
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Harish Jaiprakash
> Assignee: Harish Jaiprakash
> Priority: Blocker
>
> UnorderedPartitionedKVWriter add task to the executor, but does not wait for
> them to finish before starting the final merge. This can cause finalMerge to
> fail or write incorrect data.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
