[
https://issues.apache.org/jira/browse/HADOOP-7183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007228#comment-13007228
]
Owen O'Malley commented on HADOOP-7183:
---------------------------------------
{quote}
6881 changed it to add comparators.put there.
{quote}
Ah, right. Being too efficient. *grin*
{quote}
Making WritableComparator thread-safe by using a thread-local buffer would be a
good enhancement but isn't necessary for this fix.
{quote}
This fix is brittle precisely because the expectation is that comparators are
stateless. It looks like the shuffle doesn't reuse comparators between threads,
but it is much much easier to make them thread-safe than ensure that no one
puts in such a use case.
> WritableComparator.get should not cache comparator objects
> ----------------------------------------------------------
>
> Key: HADOOP-7183
> URL: https://issues.apache.org/jira/browse/HADOOP-7183
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Tom White
> Priority: Blocker
> Fix For: 0.20.3, 0.21.1, 0.22.0
>
> Attachments: HADOOP-7183.patch
>
>
> HADOOP-6881 modified WritableComparator.get such that the constructed
> WritableComparator gets saved back into the static map. This is fine for
> stateless comparators, but some comparators have per-instance state, and thus
> this becomes thread-unsafe and causes errors in the shuffle where multiple
> threads are doing comparisons. An example of a Comparator with per-instance
> state is WritableComparator itself.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
