[
https://issues.apache.org/jira/browse/HBASE-4367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102020#comment-13102020
]
Ted Yu commented on HBASE-4367:
-------------------------------
According to the line number above, CompactionRequest.toString() is from this
line:
{code}
LOG.trace("Inserting region in queue. " + newRequest);
{code}
I think the above code was for debugging purpose. Meaning, we can change what
CompactionRequest.toString() returns.
The Date field in CompactionRequest is used in its compareTo() method.
By examining the code, the following is always executed (starting line 61):
{code}
if (d == null) {
d = new Date();
}
{code}
I think we can replace Date field with a long field which carries timestamp.
This would satisfy its usage in compareTo() method and avoid Date.toString()
altogether.
Option 2 is to omit the date component in CompactionRequest.toString().
If any of the above options is chosen, please allow me to provide a patch.
> Deadlock in MemStore flusher due to JDK internally synchronizing on current
> thread
> ----------------------------------------------------------------------------------
>
> Key: HBASE-4367
> URL: https://issues.apache.org/jira/browse/HBASE-4367
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.90.4
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Fix For: 0.92.0
>
>
> We observed a deadlock in production between the following threads:
> - IPC handler thread holding the monitor lock on MemStoreFlusher inside
> reclaimMemStoreMemory, waiting to obtain MemStoreFlusher.lock (the reentrant
> lock member)
> - cacheFlusher thread inside flushRegion holds MemStoreFlusher.lock, and then
> calls PriorityCompactionQueue.add, which calls
> PriorityCompactionQueue.addToRegionsInQueue, which calls
> CompactionRequest.toString(), which calls Date.toString. If this occurs just
> after a GC under memory pressure, Date.toString needs to reload locale
> information (stored in a soft reference), so it calls
> ResourceBundle.loadBundle, which uses Thread.currentThread() as a
> synchronizer (see sun bug http://bugs.sun.com/view_bug.do?bug_id=6915621).
> Since the current thread is the MemStoreFlusher itself, we have a lock order
> inversion and a deadlock.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
