[
https://issues.apache.org/jira/browse/HBASE-7845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609196#comment-13609196
]
Jean-Marc Spaggiari commented on HBASE-7845:
--------------------------------------------
Here we go!
jmspaggi@hbasetest:~/hbase$ cat before.txt | grep took | grep Gaus
13/03/21 11:39:36 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 120260ms.
13/03/21 11:44:42 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119153ms.
13/03/21 11:49:49 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119066ms.
13/03/21 11:54:55 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 118865ms.
13/03/21 12:00:02 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119450ms.
13/03/21 12:05:11 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 122048ms.
13/03/21 12:10:14 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119197ms.
13/03/21 12:15:22 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119977ms.
13/03/21 12:20:32 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 121030ms.
13/03/21 12:25:38 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119880ms.
jmspaggi@hbasetest:~/hbase$ cat after.txt | grep took | grep Gaus
13/03/21 12:56:07 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119765ms.
13/03/21 13:01:14 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119730ms.
13/03/21 13:06:20 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119945ms.
13/03/21 13:11:24 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119283ms.
13/03/21 13:16:29 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 118535ms.
13/03/21 13:21:30 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 117409ms.
13/03/21 13:26:37 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119458ms.
13/03/21 13:31:43 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119178ms.
13/03/21 13:36:48 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 118427ms.
13/03/21 13:41:55 INFO hbase.HFilePerformanceEvaluation: Running
GaussianRandomReadBenchmark for 1000000 rows took 119580ms.
jmspaggi@hbasetest:~/hbase$ cat before.txt | grep took | grep Write
13/03/21 11:35:42 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1307ms.
13/03/21 11:40:51 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1333ms.
13/03/21 11:45:58 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1328ms.
13/03/21 11:51:04 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1313ms.
13/03/21 11:56:09 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1302ms.
13/03/21 12:01:15 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1322ms.
13/03/21 12:06:23 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1304ms.
13/03/21 12:11:29 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1299ms.
13/03/21 12:16:37 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1329ms.
13/03/21 12:21:45 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1354ms.
jmspaggi@hbasetest:~/hbase$ cat after.txt | grep took | grep Write
13/03/21 12:52:14 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1261ms.
13/03/21 12:57:21 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1286ms.
13/03/21 13:02:27 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1338ms.
13/03/21 13:07:33 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1309ms.
13/03/21 13:12:39 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1317ms.
13/03/21 13:17:41 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1303ms.
13/03/21 13:22:45 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1328ms.
13/03/21 13:27:51 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1302ms.
13/03/21 13:32:57 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1301ms.
13/03/21 13:38:03 INFO hbase.HFilePerformanceEvaluation: Running
SequentialWriteBenchmark for 1000000 rows took 1292ms.
jmspaggi@hbasetest:~/hbase$ cat before.txt | grep took | grep
UniformRandomSmallScan
13/03/21 11:35:46 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3989ms.
13/03/21 11:40:55 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3840ms.
13/03/21 11:46:01 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3898ms.
13/03/21 11:51:07 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3857ms.
13/03/21 11:56:13 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3912ms.
13/03/21 12:01:19 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3991ms.
13/03/21 12:06:27 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3858ms.
13/03/21 12:11:33 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3902ms.
13/03/21 12:16:41 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3898ms.
13/03/21 12:21:49 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3844ms.
jmspaggi@hbasetest:~/hbase$ cat after.txt | grep took | grep
UniformRandomSmallScan
13/03/21 12:52:18 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3853ms.
13/03/21 12:57:25 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3915ms.
13/03/21 13:02:31 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3909ms.
13/03/21 13:07:37 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3955ms.
13/03/21 13:12:43 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3852ms.
13/03/21 13:17:45 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3853ms.
13/03/21 13:22:49 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3890ms.
13/03/21 13:27:55 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3848ms.
13/03/21 13:33:01 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3881ms.
13/03/21 13:38:07 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomSmallScan for 1000000 rows took 3893ms.
jmspaggi@hbasetest:~/hbase$ cat before.txt | grep took | grep
UniformRandomReadBenchmark
13/03/21 11:37:36 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 110016ms.
13/03/21 11:42:43 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 107885ms.
13/03/21 11:47:50 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 108216ms.
13/03/21 11:52:56 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 108158ms.
13/03/21 11:58:03 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 109513ms.
13/03/21 12:03:09 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 109805ms.
13/03/21 12:08:15 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 108181ms.
13/03/21 12:13:22 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 108953ms.
13/03/21 12:18:31 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 110007ms.
13/03/21 12:23:38 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 108967ms.
jmspaggi@hbasetest:~/hbase$ cat after.txt | grep took | grep
UniformRandomReadBenchmark
13/03/21 12:54:08 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 109396ms.
13/03/21 12:59:14 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 109176ms.
13/03/21 13:04:20 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 108506ms.
13/03/21 13:09:25 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 107536ms.
13/03/21 13:14:30 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 107418ms.
13/03/21 13:19:32 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 107035ms.
13/03/21 13:24:37 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 107874ms.
13/03/21 13:29:44 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 108782ms.
13/03/21 13:34:50 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 109058ms.
13/03/21 13:39:55 INFO hbase.HFilePerformanceEvaluation: Running
UniformRandomReadBenchmark for 1000000 rows took 108650ms.
jmspaggi@hbasetest:~/hbase$ cat before.txt | grep took | grep
SequentialReadBenchmark
13/03/21 11:39:37 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 414ms.
13/03/21 11:44:42 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 447ms.
13/03/21 11:49:49 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 567ms.
13/03/21 11:54:55 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 418ms.
13/03/21 12:00:03 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 421ms.
13/03/21 12:05:11 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 414ms.
13/03/21 12:10:15 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 433ms.
13/03/21 12:15:22 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 434ms.
13/03/21 12:20:32 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 418ms.
13/03/21 12:25:38 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 432ms.
jmspaggi@hbasetest:~/hbase$ cat after.txt | grep took | grep
SequentialReadBenchmark
13/03/21 12:56:08 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 410ms.
13/03/21 13:01:14 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 418ms.
13/03/21 13:06:20 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 436ms.
13/03/21 13:11:25 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 403ms.
13/03/21 13:16:29 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 424ms.
13/03/21 13:21:30 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 423ms.
13/03/21 13:26:37 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 411ms.
13/03/21 13:31:44 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 422ms.
13/03/21 13:36:49 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 425ms.
13/03/21 13:41:55 INFO hbase.HFilePerformanceEvaluation: Running
SequentialReadBenchmark for 1000000 rows took 428ms.
Now, number wise...
Here is the average for before your patch and after your patch. The % is the
time it took for AFTER to complete compared to BEFORE. So overall, it has
always been faster. From a small 0.6% for the GaussianRandomReadBenchmark to a
big 4.5% for SequentialReadBenchmark.
GaussianRandomReadBenchmark Before 119892.6
GaussianRandomReadBenchmark After 119131 99.36%
SequentialWriteBenchmark Before 1319.1
SequentialWriteBenchmark After 1303.7 98.83%
UniformRandomallScan Before 3898.9
UniformRandomallScan After 3884.9 99.64%
UniformRandomReadBenchmark Before 108970.1
UniformRandomReadBenchmark After 108343.1 99.42%
SequentialReadBenchmark Before 439.8
SequentialReadBenchmark After 420 95.50%
> optimize hfile index key
> ------------------------
>
> Key: HBASE-7845
> URL: https://issues.apache.org/jira/browse/HBASE-7845
> Project: HBase
> Issue Type: Improvement
> Components: HFile
> Affects Versions: 0.96.0
> Reporter: Liang Xie
> Assignee: Liang Xie
> Fix For: 0.98.0
>
> Attachments: HBASE-7845.txt, HBASE-7845-v2.txt, HBASE-7845-v3.txt,
> HBASE-7845-v4.txt, HBASE-7845-v5.txt, HBASE-7845-v6.txt, HBASE-7845-v7.txt,
> HBASE-7845-v8.txt
>
>
> Leveldb uses ByteWiseComparatorImpl::FindShortestSeparator() &
> FindShortSuccessor() to reduce index key size, it would be helpful under
> special conditions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
