[
https://issues.apache.org/jira/browse/HBASE-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749148#comment-13749148
]
Jimmy Xiang commented on HBASE-9321:
------------------------------------
I was confused by the user cache stuff. This is the issue here. HConnection has
its user. Connection has the same user from HConnection. To support
impersonation, for each call, we check the current user. If the current user
is not the same user as the user in Connection/HConnection, we pass the user
over in the header as the effective user. The server side will do more
authentication/authorization. The benefits is that we don't have to create too
many connections. If we create a connection/hconnection per user, it will not
be as scalable to support impersonation.
> Contention getting the current user in RpcClient$Connection.writeRequest
> ------------------------------------------------------------------------
>
> Key: HBASE-9321
> URL: https://issues.apache.org/jira/browse/HBASE-9321
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.95.2
> Reporter: Jean-Daniel Cryans
> Fix For: 0.98.0, 0.96.0
>
>
> I've been running tests on clusters with "lots" of regions, about 400, and
> I'm seeing weird contention in the client.
> This one I see a lot, hundreds and sometimes thousands of threads are blocked
> like this:
> {noformat}
> "htable-pool4-t74" daemon prio=10 tid=0x00007f2254114000 nid=0x2a99 waiting
> for monitor entry [0x00007f21f9e94000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:466)
> - waiting to lock <0x00000000fb5ad000> (a java.lang.Class for
> org.apache.hadoop.security.UserGroupInformation)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.writeRequest(RpcClient.java:1013)
> at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1407)
> at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1634)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1691)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.multi(ClientProtos.java:27339)
> at
> org.apache.hadoop.hbase.client.MultiServerCallable.call(MultiServerCallable.java:105)
> at
> org.apache.hadoop.hbase.client.MultiServerCallable.call(MultiServerCallable.java:43)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:183)
> {noformat}
> While the holder is doing this:
> {noformat}
> "htable-pool17-t55" daemon prio=10 tid=0x00007f2244408000 nid=0x2a98 runnable
> [0x00007f21f9f95000]
> java.lang.Thread.State: RUNNABLE
> at java.security.AccessController.getStackAccessControlContext(Native
> Method)
> at java.security.AccessController.getContext(AccessController.java:487)
> at
> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:466)
> - locked <0x00000000fb5ad000> (a java.lang.Class for
> org.apache.hadoop.security.UserGroupInformation)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.writeRequest(RpcClient.java:1013)
> at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1407)
> at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1634)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1691)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.multi(ClientProtos.java:27339)
> at
> org.apache.hadoop.hbase.client.MultiServerCallable.call(MultiServerCallable.java:105)
> at
> org.apache.hadoop.hbase.client.MultiServerCallable.call(MultiServerCallable.java:43)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:183)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
