[
https://issues.apache.org/jira/browse/HBASE-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-1910:
----------------------------------
Attachment: thread.dump.gz
Attached thread dump provided by user for reference.
> IndexedRegion RPC deadlock
> --------------------------
>
> Key: HBASE-1910
> URL: https://issues.apache.org/jira/browse/HBASE-1910
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.1, 0.21.0
> Reporter: Andrew Purtell
> Fix For: 0.22.0
>
> Attachments: thread.dump.gz
>
>
> From Tatsuya Kawano up on hbase-user@
> {quote}
> 50 client threads who try to put millions of records, autoFlush(false),
> flushCommits() on every 5,000 put. After inserting about 3 million records, a
> deadlock occurred on a region server who has both the table and index regions
> loaded.
> I have attached a full thread dump of the deadlocked region server, and you
> can see IPC Server handlers are blocked in
> org.apache.hadoop.hbase.regionserver.tableindexed.IndexedRegion.updateIndex().
> I found the flowing FIXME comment on updateIndex() method, and it seems this
> is the deadlock I'm having.
> {code}
> // FIXME: This call takes place in an RPC, and requires an RPC. This makes
> for
> // a likely deadlock if the number of RPCs we are trying to serve is >= the
> // number of handler threads.
> private void updateIndex(IndexSpecification indexSpec, byte[] row,
> SortedMap<byte[], byte[]> columnValues) throws IOException {
> {code}
> I use HBase 0.20.1 and my region servers were running with 10 RPC handler
> threads on each (default).
> Maybe you can workaround this by adding more RPC handlers (increase the value
> of "hbase.regionserver.handler.count" in hbase-site.xml)
> {quote}
> Opening this issue to track the FIXME.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
