[
https://issues.apache.org/jira/browse/HBASE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089821#comment-13089821
]
stack commented on HBASE-4188:
------------------------------
So, it looks like timing difference seen by Andrew is no longer present: with
patch loads happen faster and though the timings have wide variance, after a
few runs, with patch we seem to faster.
Here is a run of MemStoreScanPerformance before I applied above patch:
{code}
stack-7:0.90 Stack$ ./bin/hbase
org.apache.hadoop.hbase.regionserver.MemStoreScanPerformance
JUnit version 4.8.1
.Loaded in 2479 ms
Scan with size 50000: 177 ms
Scan with size 50000: 185 ms
Scan with size 50000: 234 ms
Scan with size 50000: 86 ms
Scan with size 50000: 115 ms
Scan with size 50000: 92 ms
Scan with size 50000: 53 ms
Scan with size 50000: 58 ms
Scan with size 50000: 69 ms
Scan with size 50000: 61 ms
Loaded in 3134 ms
Scan with size 75000: 77 ms
Scan with size 75000: 87 ms
Scan with size 75000: 88 ms
Scan with size 75000: 84 ms
Scan with size 75000: 71 ms
Scan with size 75000: 110 ms
Scan with size 75000: 109 ms
Scan with size 75000: 100 ms
Scan with size 75000: 114 ms
Scan with size 75000: 111 ms
Loaded in 4748 ms
Scan with size 100000: 101 ms
Scan with size 100000: 85 ms
Scan with size 100000: 75 ms
Scan with size 100000: 91 ms
Scan with size 100000: 106 ms
Scan with size 100000: 84 ms
Scan with size 100000: 100 ms
Scan with size 100000: 91 ms
Scan with size 100000: 102 ms
Scan with size 100000: 81 ms
Loaded in 5904 ms
Scan with size 125000: 138 ms
Scan with size 125000: 175 ms
Scan with size 125000: 163 ms
Scan with size 125000: 145 ms
Scan with size 125000: 150 ms
Scan with size 125000: 122 ms
Scan with size 125000: 112 ms
Scan with size 125000: 118 ms
Scan with size 125000: 146 ms
Scan with size 125000: 136 ms
Loaded in 6125 ms
Scan with size 150000: 126 ms
Scan with size 150000: 106 ms
Scan with size 150000: 117 ms
Scan with size 150000: 119 ms
Scan with size 150000: 118 ms
Scan with size 150000: 118 ms
Scan with size 150000: 126 ms
Scan with size 150000: 122 ms
Scan with size 150000: 127 ms
Scan with size 150000: 117 ms
Total load time: 22390 ms (i.e:22 seconds)
Total scan time: 5597 ms (i.e:5 seconds)
Rows scanned per seconds: 893335
Rows loaded per seconds: 223313
Time: 28.787
OK (1 test)
{code}
Here is afterward:
{code}
stack-7:0.90 Stack$ ./bin/hbase
org.apache.hadoop.hbase.regionserver.MemStoreScanPerformance
JUnit version 4.8.1
.Loaded in 2293 ms
Scan with size 50000: 147 ms
Scan with size 50000: 195 ms
Scan with size 50000: 164 ms
Scan with size 50000: 88 ms
Scan with size 50000: 98 ms
Scan with size 50000: 80 ms
Scan with size 50000: 86 ms
Scan with size 50000: 57 ms
Scan with size 50000: 61 ms
Scan with size 50000: 48 ms
Loaded in 2447 ms
Scan with size 75000: 54 ms
Scan with size 75000: 94 ms
Scan with size 75000: 60 ms
Scan with size 75000: 61 ms
Scan with size 75000: 75 ms
Scan with size 75000: 59 ms
Scan with size 75000: 69 ms
Scan with size 75000: 71 ms
Scan with size 75000: 83 ms
Scan with size 75000: 57 ms
Loaded in 3381 ms
Scan with size 100000: 81 ms
Scan with size 100000: 84 ms
Scan with size 100000: 84 ms
Scan with size 100000: 85 ms
Scan with size 100000: 83 ms
Scan with size 100000: 85 ms
Scan with size 100000: 108 ms
Scan with size 100000: 78 ms
Scan with size 100000: 85 ms
Scan with size 100000: 75 ms
Loaded in 4395 ms
Scan with size 125000: 102 ms
Scan with size 125000: 104 ms
Scan with size 125000: 106 ms
Scan with size 125000: 88 ms
Scan with size 125000: 109 ms
Scan with size 125000: 99 ms
Scan with size 125000: 99 ms
Scan with size 125000: 110 ms
Scan with size 125000: 105 ms
Scan with size 125000: 105 ms
Loaded in 6552 ms
Scan with size 150000: 110 ms
Scan with size 150000: 112 ms
Scan with size 150000: 107 ms
Scan with size 150000: 126 ms
Scan with size 150000: 114 ms
Scan with size 150000: 118 ms
Scan with size 150000: 131 ms
Scan with size 150000: 110 ms
Scan with size 150000: 113 ms
Scan with size 150000: 117 ms
Total load time: 19068 ms (i.e:19 seconds)
Total scan time: 4731 ms (i.e:4 seconds)
Rows scanned per seconds: 1056859
Rows loaded per seconds: 262219
Time: 24.52
OK (1 test)
stack-7:0.90 Stack$
{code}
> Make in-memory table scanning faster, part2 (continuation of hbase-1938)
> ------------------------------------------------------------------------
>
> Key: HBASE-4188
> URL: https://issues.apache.org/jira/browse/HBASE-4188
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Reporter: stack
> Assignee: nkeywal
> Priority: Critical
> Fix For: 0.92.0
>
>
> This issue is a continuation of hbase-1938 work (That issue is closed). This
> issue is about getting the last patch posted by nkeywal over in hbase-1938
> applied (assigned nkeywal since he's done the work).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
