[
https://issues.apache.org/jira/browse/TEZ-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936097#comment-14936097
]
Rajesh Balamohan commented on TEZ-2716:
---------------------------------------
It does not cause task failures. During spills, DefaultSorter determines
whether RLE is needed or not based on the total number of keys gathered and the
number of same keys seen so far. These counters are updated in main thread.
In spill thread, these counters would not be accurately visible without proper
synchronization. This could cause invalid computation for isRLENeeded due to
thread safety issues. Patch tries to address this scenario.
> DefaultSorter.isRleNeeded not thread safe
> -----------------------------------------
>
> Key: TEZ-2716
> URL: https://issues.apache.org/jira/browse/TEZ-2716
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Siddharth Seth
> Assignee: Rajesh Balamohan
> Fix For: 0.7.1, 0.8.1
>
> Attachments: TEZ-2716.1.patch, TEZ-2716.2.patch,
> TEZ-2716.branch-0.6-and-0.5.patch
>
>
> TEZ-1997.
> Should be targeted at the same set of versions that TEZ-1997 goes into.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
