[
https://issues.apache.org/jira/browse/HBASE-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072073#comment-13072073
]
Andrew Purtell commented on HBASE-1938:
---------------------------------------
This is a representative test with 0.90 branch.
With 20110726_1938_MemStore.patch:
{noformat}
JUnit version 4.8.1
.Loaded in 1158 ms
Scan with size 50000: 61 ms
Scan with size 50000: 91 ms
Scan with size 50000: 35 ms
Scan with size 50000: 23 ms
Scan with size 50000: 24 ms
Scan with size 50000: 22 ms
Scan with size 50000: 22 ms
Scan with size 50000: 23 ms
Scan with size 50000: 22 ms
Scan with size 50000: 23 ms
Loaded in 1709 ms
Scan with size 75000: 31 ms
Scan with size 75000: 30 ms
Scan with size 75000: 30 ms
Scan with size 75000: 29 ms
Scan with size 75000: 30 ms
Scan with size 75000: 30 ms
Scan with size 75000: 30 ms
Scan with size 75000: 30 ms
Scan with size 75000: 29 ms
Scan with size 75000: 29 ms
Loaded in 2099 ms
Scan with size 100000: 36 ms
Scan with size 100000: 38 ms
Scan with size 100000: 38 ms
Scan with size 100000: 35 ms
Scan with size 100000: 35 ms
Scan with size 100000: 34 ms
Scan with size 100000: 35 ms
Scan with size 100000: 34 ms
Scan with size 100000: 34 ms
Scan with size 100000: 38 ms
Loaded in 2813 ms
Scan with size 125000: 45 ms
Scan with size 125000: 44 ms
Scan with size 125000: 53 ms
Scan with size 125000: 44 ms
Scan with size 125000: 44 ms
Scan with size 125000: 44 ms
Scan with size 125000: 44 ms
Scan with size 125000: 44 ms
Scan with size 125000: 44 ms
Scan with size 125000: 45 ms
Loaded in 3456 ms
Scan with size 150000: 54 ms
Scan with size 150000: 51 ms
Scan with size 150000: 50 ms
Scan with size 150000: 50 ms
Scan with size 150000: 50 ms
Scan with size 150000: 52 ms
Scan with size 150000: 50 ms
Scan with size 150000: 50 ms
Scan with size 150000: 51 ms
Scan with size 150000: 50 ms
Total load time: 11235 ms (i.e:11 seconds)
Total scan time: 1959 ms (i.e:1 seconds)
Rows scanned per seconds: 2552322
Rows loaded per seconds: 445037
Time: 13.515
OK (1 test)
{noformat}
Without 20110726_1938_MemStore.patch:
{noformat}
JUnit version 4.8.1
.Loaded in 1197 ms
Scan with size 50000: 51 ms
Scan with size 50000: 119 ms
Scan with size 50000: 21 ms
Scan with size 50000: 22 ms
Scan with size 50000: 22 ms
Scan with size 50000: 22 ms
Scan with size 50000: 21 ms
Scan with size 50000: 22 ms
Scan with size 50000: 22 ms
Scan with size 50000: 22 ms
Loaded in 1496 ms
Scan with size 75000: 31 ms
Scan with size 75000: 32 ms
Scan with size 75000: 29 ms
Scan with size 75000: 29 ms
Scan with size 75000: 30 ms
Scan with size 75000: 30 ms
Scan with size 75000: 37 ms
Scan with size 75000: 30 ms
Scan with size 75000: 30 ms
Scan with size 75000: 30 ms
Loaded in 2160 ms
Scan with size 100000: 33 ms
Scan with size 100000: 32 ms
Scan with size 100000: 33 ms
Scan with size 100000: 32 ms
Scan with size 100000: 32 ms
Scan with size 100000: 33 ms
Scan with size 100000: 32 ms
Scan with size 100000: 33 ms
Scan with size 100000: 32 ms
Scan with size 100000: 34 ms
Loaded in 2720 ms
Scan with size 125000: 42 ms
Scan with size 125000: 51 ms
Scan with size 125000: 41 ms
Scan with size 125000: 42 ms
Scan with size 125000: 41 ms
Scan with size 125000: 42 ms
Scan with size 125000: 42 ms
Scan with size 125000: 41 ms
Scan with size 125000: 41 ms
Scan with size 125000: 43 ms
Loaded in 3518 ms
Scan with size 150000: 49 ms
Scan with size 150000: 49 ms
Scan with size 150000: 50 ms
Scan with size 150000: 49 ms
Scan with size 150000: 50 ms
Scan with size 150000: 49 ms
Scan with size 150000: 49 ms
Scan with size 150000: 50 ms
Scan with size 150000: 49 ms
Scan with size 150000: 49 ms
Total load time: 11091 ms (i.e:11 seconds)
Total scan time: 1895 ms (i.e:1 seconds)
Rows scanned per seconds: 2638522
Rows loaded per seconds: 450815
Time: 13.375
OK (1 test)
{noformat}
> Make in-memory table scanning faster
> ------------------------------------
>
> Key: HBASE-1938
> URL: https://issues.apache.org/jira/browse/HBASE-1938
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Reporter: stack
> Assignee: nkeywal
> Priority: Blocker
> Fix For: 0.90.4, 0.92.0
>
> Attachments: 20110726_1938_KeyValueSkipListSet.patch,
> 20110726_1938_MemStore.patch, 20110726_1938_MemStoreScanPerformance.java,
> MemStoreScanPerformance.java, MemStoreScanPerformance.java,
> caching-keylength-in-kv.patch, test.patch
>
>
> This issue is about profiling hbase to see if I can make hbase scans run
> faster when all is up in memory. Talking to some users, they are seeing
> about 1/4 million rows a second. It should be able to go faster than this
> (Scanning an array of objects, they can do about 4-5x this).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
