[
https://issues.apache.org/jira/browse/TEZ-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16013767#comment-16013767
]
Harish Jaiprakash commented on TEZ-3701:
----------------------------------------
Thanks [~rajesh.balamohan].
There is a race b/w SpillCallable availableSlots.release() vs
availableSlots.acquire()/availableSlots.tryAcquire() in scheduleSpill. This
happens because the future runs in the same thread of the executor service and
will take a small amount of time after release to free the thread. We can get
rejected exception in those cases.
It would be nice to wrap only availableSlots.acquire() in a try catch and
return early. This is just to ensure that in future nothing inside it will
throw interrupted exception and forget to release availableSlots.
> UnorderedPartitionedKVWriter - issues with parallel Deflater usage,
> synchronousqueue in threadpool
> --------------------------------------------------------------------------------------------------
>
> Key: TEZ-3701
> URL: https://issues.apache.org/jira/browse/TEZ-3701
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Harish Jaiprakash
> Assignee: Rajesh Balamohan
> Priority: Blocker
> Attachments: TEZ-3701.2.patch, TEZ-3701.3.patch, TEZ-3701.4.patch
>
>
> UnorderedPartitionedKVWriter add task to the executor, but does not wait for
> them to finish before starting the final merge. This can cause finalMerge to
> fail or write incorrect data.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
