[
https://issues.apache.org/jira/browse/HBASE-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877460#action_12877460
]
HBase Review Board commented on HBASE-2468:
-------------------------------------------
Message from: "Jonathan Gray" <[email protected]>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/98/#review176
-----------------------------------------------------------
This looks great! I think we should think more about where to expose this, and
also think about using more of a get/set API to reduce the method calls and
make it look more like the other client APIs.
src/main/java/org/apache/hadoop/hbase/client/HConnection.java
<http://review.hbase.org/r/98/#comment841>
+1 on collapsing with a boolean. setRegionCachePrefetch(table, enable)?
Seems self descriptive and with a couple lines of javadoc should be clear.
src/main/java/org/apache/hadoop/hbase/client/HTable.java
<http://review.hbase.org/r/98/#comment840>
I guess these are static because of how HTables all share a single HCM per
conf. The setting of prefetching is set at the HCM level not the HTable level,
however clients are usually not exposed to HCM and only deal with HTable.
We should probably make it clear in the javadoc for these methods that they
apply to all HTable instances, though that may be clear from being static.
Maybe since these are more advanced calls, they shouldn't be in HTable? If
we provide proper documentation, it should be easy enough for a user to grab
the HCM and apply the config at that level?
src/main/java/org/apache/hadoop/hbase/client/HTable.java
<http://review.hbase.org/r/98/#comment842>
Should we also make these methods (if we even leave it in HTable) more like
setRegionCachePrefetch / getRegionCachePrefetch? HTable is the core client
class so the less noise we add the better. We have other methods in client
APIs of the get/set form like Scan.setCacheBlocks and Put.setWriteToWAL
src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java
<http://review.hbase.org/r/98/#comment843>
Hmm, wouldn't that be Math.min then? If rowLimit = 5 and scanner.caching =
100, we want to only do 5?
- Jonathan
> Improvements to prewarm META cache on clients
> ---------------------------------------------
>
> Key: HBASE-2468
> URL: https://issues.apache.org/jira/browse/HBASE-2468
> Project: HBase
> Issue Type: Improvement
> Components: client
> Reporter: Todd Lipcon
> Assignee: Mingjie Lai
> Fix For: 0.21.0
>
> Attachments: HBASE-2468-trunk.patch
>
>
> A couple different use cases cause storms of reads to META during startup.
> For example, a large MR job will cause each map task to hit meta since it
> starts with an empty cache.
> A couple possible improvements have been proposed:
> - MR jobs could ship a copy of META for the table in the DistributedCache
> - Clients could prewarm cache by doing a large scan of all the meta for the
> table instead of random reads for each miss
> - Each miss could fetch ahead some number of rows in META
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
