[
https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Douglas updated HADOOP-3617:
----------------------------------
Status: Open (was: Patch Available)
There remains a (practically unrealizable) possibility of deadlock in this
patch. Given the spill thread:
{noformat}
while (true) {
synchronized (spillLock) {
spillLock.notify();
while (kvstart == kvend) {
spillLock.wait();
}
}
try {
sortAndSpill();
} catch (Throwable e) {
sortSpillException = e;
} finally {
synchronized (spillLock) {
// adjust indices
}
}
}
{noformat}
It's possible to acquire the lock in the finally block, release it, block at
the front of the loop as the other thread acquires the lock and signals before
the spill thread can reacquire the lock at the top of the loop. In practice,
the only possibility of deadlock would be from the collection thread acquiring
the lock in a write that cannot be satisfied without spilling- again-
immediately. Still, it needs to be fixed.
What this attempts to effect seems ill-suited to block-level synchronization
and will either require some additional state or more fine-grained locking
(i.e. java.util.concurrent.lock).
> Writes from map serialization include redundant checks for accounting space
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3617
> URL: https://issues.apache.org/jira/browse/HADOOP-3617
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Fix For: 0.19.0
>
> Attachments: 3617-0.patch, 3617-1.patch
>
>
> In BlockingBuffer::write, the collector verifies that there is sufficient
> space to hold metadata when it returns to MapOutputBuffer::collect. This
> check is redundant for serialized records that make more than one call to
> write.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
