[
https://issues.apache.org/jira/browse/HBASE-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145499#comment-15145499
]
stack commented on HBASE-15240:
-------------------------------
bq. Not saying it shouldn't be improved, but prefetch is only supposed to be
used for IN_MEMORY or equivalent use cases where you want it all or just about
all preloaded and you know there's enough room (or in memory doesn't make sense
then either)
So, even for above, needs a bit of work. Cuts off at 100blocks from each
file... perhaps from all files... and then it keeps reading the file even after
it has gone past the limit. No harm. Will put up some patches.
bq. And best effort / background fetch is intended, so even though we want
preloading we are not overrunning IO bandwidth with that activity along with
other reading and writing. I think some of this is discussed on the issue that
introduced it, in the discussion there
Sure to this but looks like with current configs, it is easy to overwhelm
prefetch; the no-wait on cache and 50ms to get something into the cache are
good (maybe even high) for low latency serving but for prefetch, it means we
load a few hundred MB only from the 20GB file.... Maybe I need to make queues
bigger and add more serializing threads. We probably want to do better than
that at least for the case where a fella is trying to test bucketcache at
scale.... Should be switches so they can get the cache loaded fast. Or even
for the production in-memory case... we don't want it to flake out too easily
when prefetch is set.
Will put some patches....
Thanks.
> Go Big BucketCache Fixes
> ------------------------
>
> Key: HBASE-15240
> URL: https://issues.apache.org/jira/browse/HBASE-15240
> Project: HBase
> Issue Type: Umbrella
> Components: BucketCache
> Reporter: stack
> Assignee: stack
> Attachments: Screen Shot 2016-02-10 at 6.27.26 AM.png
>
>
> Umbrella issue to which we will attach issues that prevent bucketcache going
> big; there's a few.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
