[
https://issues.apache.org/jira/browse/HBASE-18500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121630#comment-16121630
]
Anoop Sam John commented on HBASE-18500:
----------------------------------------
Thanks for the nice debug. Things are clear now.
This Table.put() happens within the RS. We are already in AccessController.
Means already there was an RPC happened to server (Actual grant call from
client side). At RpcServer we have a ThreadLocal where we keep the CallContext
and inside that the current RPC called user info is set. The table on which put
was called is created via the RegionCP env and that uses a special Connection.
The normal RPC channel will be by passed here means there would have no further
contact on to the RpcServer. So the ThreadLocal is never getting reset. We ran
the new put as a super user (User.runAsLoginUser where the login user is the
user who started RS process) but still as per the RPC context it is the old
user. When this patch changed the Table.put(Put) flow so that there is no more
AsyncProcess involvement, the same thread executing the put. So the Rpc context
ThreadLocal is still accessible and we see the user as old Rpc user. When
AsyncProcess was used, the execute happen via another thread from pool and so
old ThreadLocal variable is not accessible and so it looks as if no Rpc context
and we were relying on the super user who starts the RS process.
We can solve this particular issue in diff ways. But this investigation raises
few Qs.
1. We provide CPs a way to make Table ops. The Table can be obtained from the
env and may be the flow will get RPC bypassed if the calling region on same RS.
In that case if the Table.put(Put) is called we will see the old RPC user in
the second region call path also. But if it was AsyncProcess involved flow, we
wont see. How/whether we should make possible to get the Rpc context in the
second region flow also? To know the user. And we have to solve this
incosistency based on API usage any way.
2. When doing a runAsUser, how/whether we should reset the Rpc context to have
the new user.
We should address it via some other jira. As of here, just do as what u did in
patch with a TODO to fix the issue.
> Performance issue: Don't use BufferedMutator for HTable's put method
> --------------------------------------------------------------------
>
> Key: HBASE-18500
> URL: https://issues.apache.org/jira/browse/HBASE-18500
> Project: HBase
> Issue Type: Improvement
> Reporter: Guanghao Zhang
> Assignee: Guanghao Zhang
> Attachments: HBASE-18500-v1.patch, HBASE-18500-v2.patch,
> HBASE-18500-v3.patch, HBASE-18500-v4.patch, HBASE-18500-v5.patch,
> HBASE-18500-v5.patch, HBASE-18500-v5.patch
>
>
> Copied the test result from HBASE-17994.
> Run start-hbase.sh in my local computer and use the default config to test
> with PE tool.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=100000
> --nomapred --autoFlush=True randomWrite 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=100000
> --nomapred --autoFlush=True asyncRandomWrite 1
> {code}
> Mean latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 164.39 | 161.22 | 164.78 | 140.61 | 151.69 |
> | asyncRandomWrite | 122.29 | 125.58 | 122.23 | 113.18 | 123.02 |
> 50th latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 130.00 | 125.00 | 123.00 | 112.00 | 121.00 |
> | asyncRandomWrite | 95.00 | 97.00 | 95.00 | 88.00 | 95.00 |
> 99th latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 600.00 | 600.00 | 650.00 | 404.00 | 425.00 |
> | asyncRandomWrite | 339.00 | 327.00 | 297.00 | 311.00 | 318.00 |
> In our internal 0.98 branch, the PE test result shows the async write has the
> almost same latency with the blocking write. But for master branch, the
> result shows the async write has better latency than the blocking client.
> Take a look about the code, I thought the difference is the BufferedMutator.
> For master branch, HTable don't have a write buffer and all write request
> will be flushed directly. And user can use BufferedMutator when user want to
> perform client-side buffering of writes. For the performance issue
> (autoFlush=True), I thought we can use rpc caller directly in HTable's put
> method. Thanks.
> Review: https://reviews.apache.org/r/61454/
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
