[ https://issues.apache.org/jira/browse/HBASE-18500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121087#comment-16121087 ]
Guanghao Zhang edited comment on HBASE-18500 at 8/10/17 5:44 AM: ----------------------------------------------------------------- bq. t.put(Collections.singletonList(p)); When grant user permission, it will put a record to ACL. The old implementation use BufferMutator, so it will send a multi request. And use AccessController.preBatchMutate to check permission when put a record. When HTable doesn't use BufferMutator, it will send a mutate request and use AccessController.prePut to check permission. So I change to use put(List<put>), this will send multi request, too. Then it will same with before. I thought AccessController may have some bugs which need to dig more. But it is not related to this issue? bq. UTIL.getConfiguration().setInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER, 2); The default retries number is 35. So it will wait a long time when there are exception in ut. I thought we should set this to the default HBaseTestingUtility config in the future. was (Author: zghaobac): bq. t.put(Collections.singletonList(p)); When grant user permission, it will put a record to ACL. The old implementation use BufferMutator, so it will send a multi request. And use AccessControlLists.preBatchMutate to check permission when put a record. When HTable doesn't use BufferMutator, it will send a mutate request and use AccessControlLists.prePut to check permission. So I change to use put(List<put>), this will send multi request, too. Then it will same with before. I thought AccessControlLists may have some bugs which need to dig more. But it is not related to this issue? bq. UTIL.getConfiguration().setInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER, 2); The default retries number is 35. So it will wait a long time when there are exception in ut. I thought we should set this to the default HBaseTestingUtility config in the future. > Performance issue: Don't use BufferedMutator for HTable's put method > -------------------------------------------------------------------- > > Key: HBASE-18500 > URL: https://issues.apache.org/jira/browse/HBASE-18500 > Project: HBase > Issue Type: Improvement > Reporter: Guanghao Zhang > Assignee: Guanghao Zhang > Attachments: HBASE-18500-v1.patch, HBASE-18500-v2.patch, > HBASE-18500-v3.patch, HBASE-18500-v4.patch, HBASE-18500-v5.patch, > HBASE-18500-v5.patch, HBASE-18500-v5.patch > > > Copied the test result from HBASE-17994. > Run start-hbase.sh in my local computer and use the default config to test > with PE tool. > {code} > ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=100000 > --nomapred --autoFlush=True randomWrite 1 > ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=100000 > --nomapred --autoFlush=True asyncRandomWrite 1 > {code} > Mean latency test result. > || || Test1 || Test2 || Test3 || Test4 || Test5 || > | randomWrite | 164.39 | 161.22 | 164.78 | 140.61 | 151.69 | > | asyncRandomWrite | 122.29 | 125.58 | 122.23 | 113.18 | 123.02 | > 50th latency test result. > || || Test1 || Test2 || Test3 || Test4 || Test5 || > | randomWrite | 130.00 | 125.00 | 123.00 | 112.00 | 121.00 | > | asyncRandomWrite | 95.00 | 97.00 | 95.00 | 88.00 | 95.00 | > 99th latency test result. > || || Test1 || Test2 || Test3 || Test4 || Test5 || > | randomWrite | 600.00 | 600.00 | 650.00 | 404.00 | 425.00 | > | asyncRandomWrite | 339.00 | 327.00 | 297.00 | 311.00 | 318.00 | > In our internal 0.98 branch, the PE test result shows the async write has the > almost same latency with the blocking write. But for master branch, the > result shows the async write has better latency than the blocking client. > Take a look about the code, I thought the difference is the BufferedMutator. > For master branch, HTable don't have a write buffer and all write request > will be flushed directly. And user can use BufferedMutator when user want to > perform client-side buffering of writes. For the performance issue > (autoFlush=True), I thought we can use rpc caller directly in HTable's put > method. Thanks. > Review: https://reviews.apache.org/r/61454/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)