[
https://issues.apache.org/jira/browse/HBASE-17747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15907134#comment-15907134
]
Yu Li commented on HBASE-17747:
-------------------------------
Ok, here comes the E2E test report through YCSB (with 100% cache hit ratio):
|| Case || Off-heap || Round || Throughput || AverageThroughput||
| w/o patch | No | 1 | 275652.627079| |
| | | 2 | 248664.333337| |
| | | 3 | 264297.580882| |
| | | 4 | 253694.307118| |
| | | | | 260577.212104 |
| w/ patch | No | 1 | 268122.883638| |
| | | 2 | 284483.969107| |
| | | 3 | 283008.465412| |
| | | 4 | 268476.579267| |
| | | | | 276022.974356 (+5.9%) |
| w/o patch | Yes | 1 | 199073.764764| |
| | | 2 | 210060.769909| |
| | | 3 | 216729.412051| |
| | | 4 | 221610.155932| |
| | | | | 211868.525664 |
| w/ patch | Yes | 1 | 217398.730766| |
| | | 2 | 232581.157272| |
| | | 3 | 225765.026296| |
| | | 4 | 229071.140011| |
| | | | | 226204.013586 (+6.8%) |
>From the result we could see with soft reference we get a better performance.
More details about testing configuration:
{noformat}
YCSB:
* 4 physical nodes, 8 ycsb processes on each node
* recordcount=11000000
* fieldcount=1
* fieldlength=1024 (10.8GB overall)
* threadcount=8
* maxexecutiontime=300
* table schema: {NAME => 'cf', DATA_BLOCK_ENCODING => 'DIFF', VERSIONS=> '1',
COMPRESSION => 'SNAPPY', IN_MEMORY => 'false', BLOCKCACHE => 'true'},{SPLITS =>
(1..9).map {|i| "user#{1000+i*(9999-1000)/9}"},
MAX_FILESIZE=>'223372036854775807',DURABILITY=>'SYNC_WAL',METADATA =>
{'hbase.hstore.block.storage.policy' => 'ALL_SSD'}}
HBase:
* One single regionserver, 3 datanode
* base commit SHA (w/o patch): 31bc94ae60
* hbase.regionserver.handler.count=192
* On-heap: -Xmx42g, hfile.block.cache.size=0.3
* Off-heap: hbase.bucketcache.size=12288,
hbase.bucketcache.writer.queuelength=64, hbase.bucketcache.writer.threads=3
{noformat}
> Support both weak and soft object pool
> --------------------------------------
>
> Key: HBASE-17747
> URL: https://issues.apache.org/jira/browse/HBASE-17747
> Project: HBase
> Issue Type: Improvement
> Reporter: Yu Li
> Assignee: Yu Li
> Attachments: HBASE-17747.patch, HBASE-17747.v2.patch,
> HBASE-17747.v3.patch
>
>
> During YCSB testing on embedded mode after HBASE-17744, we found that under
> high read load GC is quite severe even with offheap L2 cache. After some
> investigation, we found it's caused by using weak reference for
> IdReadWriteLock. In embedded mode the read is so quick that the lock might
> already get promoted to the old generation when the weak reference is
> cleared, which causes dirty card table thus slowing YGC.
> So we proposed to use soft reference for this IdReadWriteLock used in cache,
> which won't get cleared until JVM memory is not enough, and could resolve the
> issue mentioned above. What's more, we propose to extend the WeakObjectPool
> to be more generate to support both weak and soft reference.
> Note that this issue only emerges under embedded mode with DirectOperator, in
> which case all costs on the wire is removed thus produces extremely high
> workloads.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
