[
https://issues.apache.org/jira/browse/HBASE-21065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513580#comment-17513580
]
Bryan Beaudreault commented on HBASE-21065:
-------------------------------------------
Just as an update, we integrated this patch into our environment and also
updated all of our existing hbase2 clusters to have this setup. That's
currently only about 70 smallish clusters, but we'll be getting to our bigger
clusters soon. Will loop back if we encounter any issues, but until then assume
that no news is good :)
> Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters on meta while we
> are at it)
> -------------------------------------------------------------------------------------
>
> Key: HBASE-21065
> URL: https://issues.apache.org/jira/browse/HBASE-21065
> Project: HBase
> Issue Type: Improvement
> Components: meta, Performance
> Reporter: Michael Stack
> Assignee: Andrew Kyle Purtell
> Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.6.0
>
>
> Some users end up hitting meta hard. Bulk is probably because our client goes
> to meta too often, and the real 'fix' for a saturated meta is splitting it,
> but the encoding that came in with HBASE-16213, ROW_INDEX_V1, could help in
> the near term. It adds an index on hfile blocks and helped improve random
> reads against user-space tables (less compares as we used index to go direct
> to requested Cells rather than look at each Cell in turn until we found what
> we wanted -- see RN on HBASE-16213 for citation).
> I also noticed code-reading that we don't enable blooms on hbase:meta tables;
> that could save some CPU and speed things up a bit too:
> {code}
> // Disable blooms for meta. Needs work. Seems to mess w/
> getClosestOrBefore.
> .setBloomFilterType(BloomType.NONE)
> {code}
> This issue is about doing a bit of perf compare of encoding *on* vs current
> default (and will check diff in size of indexed blocks).
> Meta access is mostly random-read I believe (A review of a user's access
> showed this so at least for their workload). The nice addition, HBASE-19722
> Meta query statistics metrics source, would help verify if it saw some usage
> on a prod cluster.
> If all is good, I'd like to make a small patch, one that could be easily
> backported, with minimal changes in it.
> As is, its all a little awkward as the meta table schema is hard-coded and
> meta is immutable -- stuff we'll have to fix if we want to split meta -- so
> in the meantime it requires a code change to enable (and a backport of
> HBASE-16213 -- this patch is in 1.4.0 only currently, perhaps that is
> enough). Code change to enable is small:
> {code}
> diff --git
> a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
>
> b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> index 28c7ec3c2f..8f08f94dc1 100644
> ---
> a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> +++
> b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> @@ -160,6 +160,7 @@ public class FSTableDescriptors implements
> TableDescriptors {
> .setScope(HConstants.REPLICATION_SCOPE_LOCAL)
> // Disable blooms for meta. Needs work. Seems to mess w/
> getClosestOrBefore.
> .setBloomFilterType(BloomType.NONE)
> +
> .setDataBlockEncoding(org.apache.hadoop.hbase.io.encoding.DataBlockEncoding.ROW_INDEX_V1)
> .build())
>
> .setColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(HConstants.TABLE_FAMILY)
> .setMaxVersions(conf.getInt(HConstants.HBASE_META_VERSIONS,
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
