Repository: hbase Updated Branches: refs/heads/master da01d2f53 -> 7335f5f87
Edit of doc on block cache adding reference to blockcache doc with findings recommending when to use offheap blockcache Project: http://git-wip-us.apache.org/repos/asf/hbase/repo Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/7335f5f8 Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/7335f5f8 Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/7335f5f8 Branch: refs/heads/master Commit: 7335f5f87ddc066b5f12a396ccf44448736a217d Parents: da01d2f Author: stack <[email protected]> Authored: Thu Aug 7 22:02:48 2014 -0700 Committer: stack <[email protected]> Committed: Thu Aug 7 22:03:02 2014 -0700 ---------------------------------------------------------------------- src/main/docbkx/book.xml | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hbase/blob/7335f5f8/src/main/docbkx/book.xml ---------------------------------------------------------------------- diff --git a/src/main/docbkx/book.xml b/src/main/docbkx/book.xml index 0234f85..21e0af5 100644 --- a/src/main/docbkx/book.xml +++ b/src/main/docbkx/book.xml @@ -1950,7 +1950,7 @@ rs.close(); option, and configuration options for each.</para> <note><title>Block Cache Reporting: UI</title> - <para>See the RegionServer UI for detail on caching deploy. Since HBase 1.0, the + <para>See the RegionServer UI for detail on caching deploy. Since HBase-0.98.4, the Block Cache detail has been significantly extended showing configurations, sizings, current usage, and even detail on block counts and types.</para> </note> @@ -1972,7 +1972,11 @@ rs.close(); you'd use BucketCache, so your latencies are less erratic and to mitigate GCs and heap fragmentation. See Nick Dimiduk's <link xlink:href="http://www.n10k.com/blog/blockcache-101/";>BlockCache 101</link> for - comparisons running onheap vs offheap tests. + comparisons running onheap vs offheap tests. Also see + <link xlink:href="http://people.apache.org/~stack/bc/";>Comparing BlockCache Deploys</link> + which finds that if your dataset fits inside your LruBlockCache deploy, use it otherwise + if you are experiencing cache churn (or you want your cache to exist beyond the + vagaries of java GC), use BucketCache. </para> <para>When you enable BucketCache, you are enabling a two tier caching @@ -2208,11 +2212,11 @@ rs.close(); </para> <step> <para>First, edit the RegionServer's <filename>hbase-env.sh</filename> and set - -XX:MaxDirectMemorySize to a value greater than the offheap size wanted, in + <varname>-XX:MaxDirectMemorySize</varname> to a value greater than the offheap size wanted, in this case, 4 GB (expressed as 4G). Lets set it to 5G. That'll be 4G for our offheap cache and 1G for any other uses of offheap memory (there are other users of offheap memory other than BlockCache; e.g. DFSClient - in RegionServer can make use of offheap memory).</para> + in RegionServer can make use of offheap memory). See <xref linkend="direct.memory" />.</para> <programlisting>-XX:MaxDirectMemorySize=5G</programlisting> </step> <step> @@ -2258,6 +2262,10 @@ rs.close(); easy to quantify; it is the number of open hfiles * <varname>hbase.dfs.client.read.shortcircuit.buffer.size</varname> where hbase.dfs.client.read.shortcircuit.buffer.size is set to 128k in HBase -- see <filename>hbase-default.xml</filename> default configurations). + Direct memory, which is part of the Java process heap, is separate from the object + heap allocated by -Xmx. The value allocated by MaxDirectMemorySize must not exceed + physical RAM, and is likely to be less than the total available RAM due to other + memory requirements and system constraints. </para> <para>You can see how much memory -- onheap and offheap/direct -- a RegionServer is configured to use and how much it is using at any one time by looking at the
