[
https://issues.apache.org/jira/browse/HBASE-17747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923555#comment-15923555
]
Yu Li commented on HBASE-17747:
-------------------------------
And some direct answers to above questions:
bq. With a lower pressure will it cause the pool to be very very large as we do
not need to reclaim it?
Yes, possibly. But with no harm? Maybe concerns on causing fragment? I checked
with [~haoran] -- our JVM source developer and expert here -- that soft
reference is assured to be cleared before OOM if the memory is not enough, so a
full GC could perfectly resolve this.
bq. If you are running this in prod with loads of churn, would be good to know.
This is still not running in prod, but will do soon. Will keep an eye on this
and open new issue if observed any issue. As the original author of
{{IdReadWriteLock}}, it's my responsibility to make sure it could work well for
sure (smile).
bq. And have you test it with G1?
No I haven't tested with G1, but I think if G1 cannot deal with soft reference
well, it's some kind of bug in G1 rather than use error of soft reference.
> Support both weak and soft object pool
> --------------------------------------
>
> Key: HBASE-17747
> URL: https://issues.apache.org/jira/browse/HBASE-17747
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 2.0
> Reporter: Yu Li
> Assignee: Yu Li
> Fix For: 2.0
>
> Attachments: HBASE-17747.patch, HBASE-17747.v2.patch,
> HBASE-17747.v3.patch
>
>
> During YCSB testing on embedded mode after HBASE-17744, we found that under
> high read load GC is quite severe even with offheap L2 cache. After some
> investigation, we found it's caused by using weak reference in
> {{IdReadWriteLock}}. In embedded mode the read is so quick that the lock
> might already get promoted to the old generation when the weak reference is
> cleared, which causes dirty card table (old reference get removed and new
> lock object set into {{referenceCache}}, see {{WeakObjectPool#get}}) thus
> slowing YGC. In distributed mode there'll also be more lock object created
> with weak reference than soft reference that slowing down the processing.
> So we proposed to use soft reference for this {{IdReadWriteLock}} used in
> cache, which won't get cleared until JVM memory is not enough, and could
> resolve the issue mentioned above. What's more, we propose to extend the
> {{WeakObjectPool}} to be more generate to support both weak and soft
> reference.
> Note that the GC issue only emerges under embedded mode with DirectOperator,
> in which case all costs on the wire is removed thus produces extremely high
> concurrency.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
