[
https://issues.apache.org/jira/browse/HBASE-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906631#action_12906631
]
Todd Lipcon commented on HBASE-2964:
------------------------------------
Fixing this is a little tricky. We could short-circuit the IPC path when
detecting that a region is hosted in the same process, and thus avoid going
through handlers (this is what the datanode does in the block recovery code).
However, you still can have a situation where two regionservers are trying to
talk to each other and end up in a deadlock.
Another option is to add a timeout to these RPCs, abort the split and try again
later if it fails.
Another thing that might help is to have the start of the split transaction
flag the table as "going offline", and before taking the readlock, other
accessors of the table can check for this case and immediately throw NSRE
rather than blocking once the split is in progress.
> Deadlock when RS tries to RPC to itself inside SplitTransaction
> ---------------------------------------------------------------
>
> Key: HBASE-2964
> URL: https://issues.apache.org/jira/browse/HBASE-2964
> Project: HBase
> Issue Type: Bug
> Components: ipc, regionserver
> Affects Versions: 0.90.0
> Reporter: Todd Lipcon
> Priority: Blocker
>
> In testing the 0.89.20100830 rc, I ran into a deadlock with the following
> situation:
> - All of the IPC Handler threads are blocked on the region lock, which is
> held by CompactSplitThread.
> - CompactSplitThread is in the process of trying to edit META to create the
> offline parent. META happens to be on the same server as is executing the
> split.
> Therefore, the CompactSplitThread is trying to connect back to itself, but
> all of the handler threads are blocked, so the IPC never happens. Thus, the
> entire RS gets deadlocked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
