[09/26] hbase git commit: HBASE-15124 Document the new 'normalization' feature in refguid

2017-12-22 Thread zhangduo 
HBASE-15124 Document the new 'normalization' feature in refguid


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/070c2ee7
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/070c2ee7
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/070c2ee7

Branch: refs/heads/HBASE-19397
Commit: 070c2ee7dd371157c1c242fe28b91f76ba47c5c5
Parents: 37bf54a
Author: Michael Stack 
Authored: Thu Dec 21 22:11:12 2017 -0800
Committer: Michael Stack 
Committed: Thu Dec 21 22:47:25 2017 -0800

--
 src/main/asciidoc/_chapters/ops_mgt.adoc | 136 ++
 1 file changed, 136 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/hbase/blob/070c2ee7/src/main/asciidoc/_chapters/ops_mgt.adoc
--
diff --git a/src/main/asciidoc/_chapters/ops_mgt.adoc 
b/src/main/asciidoc/_chapters/ops_mgt.adoc
index 7b0f89b..f5fa456 100644
--- a/src/main/asciidoc/_chapters/ops_mgt.adoc
+++ b/src/main/asciidoc/_chapters/ops_mgt.adoc
@@ -2704,4 +2704,140 @@ Viewing the Master log will give you insight on rsgroup 
operation.
 If it appears stuck, restart the Master process.
 
 
+[[normalizer]]
+== Region Normalizer
+
+The Region Normalizer tries to make Regions all in a table about the same in 
size.
+It does this by finding a rough average. Any region that is larger than twice 
this
+size is split. Any region that is much smaller is merged into an adjacent 
region.
+It is good to run the Normalizer on occasion on a down time after the cluster 
has
+been running a while or say after a burst of activity such as a large delete.
+
+(The bulk of the below detail was copied wholesale from the blog by Romil 
Choksi at
+link:https://community.hortonworks.com/articles/54987/hbase-region-normalizer.html[HBase
 Region Normalizer])
+
+The Region Normalizer is feature available since HBase-1.2. It runs a set of
+pre-calculated merge/split actions to resize regions that are either too
+large or too small compared to the average region size for a given table. 
Region
+Normalizer when invoked computes a normalization 'plan' for all of the tables 
in
+HBase. System tables (such as hbase:meta, hbase:namespace, Phoenix system 
tables
+etc) and user tables with normalization disabled are ignored while computing 
the
+plan. For normalization enabled tables, normalization plan is carried out in
+parallel across multiple tables.
+
+Normalizer can be enabled or disabled globally for the entire cluster using the
+‘normalizer_switch’ command in the HBase shell. Normalization can also be
+controlled on a per table basis, which is disabled by default when a table is
+created. Normalization for a table can be enabled or disabled by setting the
+NORMALIZATION_ENABLED table attribute to true or false.
+
+To check normalizer status and enable/disable normalizer
 
+[source,bash]
+
+hbase(main):001:0> normalizer_enabled
+true 
+0 row(s) in 0.4870 seconds
+ 
+hbase(main):002:0> normalizer_switch false
+true 
+0 row(s) in 0.0640 seconds
+ 
+hbase(main):003:0> normalizer_enabled
+false 
+0 row(s) in 0.0120 seconds
+ 
+hbase(main):004:0> normalizer_switch true
+false
+0 row(s) in 0.0200 seconds
+ 
+hbase(main):005:0> normalizer_enabled
+true
+0 row(s) in 0.0090 seconds
+
+
+When enabled, Normalizer is invoked in the background every 5 mins (by 
default),
+which can be configured using `hbase.normalization.period` in `hbase-site.xml`.
+Normalizer can also be invoked manually/programmatically at will using HBase 
shell’s
+`normalize` command. HBase by default uses `SimpleRegionNormalizer`, but users 
can
+design their own normalizer as long as they implement the RegionNormalizer 
Interface.
+Details about the logic used by `SimpleRegionNormalizer` to compute its 
normalization
+plan can be found 
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.html[here].
+
+The below example shows a normalization plan being computed for an user table, 
and
+merge action being taken as a result of the normalization plan computed by 
SimpleRegionNormalizer.
+
+Consider an user table with some pre-split regions having 3 equally large 
regions
+(about 100K rows) and 1 relatively small region (about 25K rows). Following is 
the
+snippet from an hbase meta table scan showing each of the pre-split regions 
for 
+the user table.
+
+
+table_p8ddpd6q5z,,1469494305548.68b9892220865cb6048 column=info:regioninfo, 
timestamp=1469494306375, value={ENCODED => 68b9892220865cb604809c950d1adf48, 
NAME => 'table_p8ddpd6q5z,,1469494305548.68b989222 09c950d1adf48.   
0865cb604809c950d1adf48.', STARTKEY => '', ENDKEY => '1'} 
+ 
+table_p8ddpd6q5z,1,1469494317178.867b77333bdc75a028 column=info:regioninfo,

hbase git commit: HBASE-15124 Document the new 'normalization' feature in refguid

2017-12-21 Thread stack 
Repository: hbase
Updated Branches:
  refs/heads/master 37bf54a51 -> 070c2ee7d


HBASE-15124 Document the new 'normalization' feature in refguid


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/070c2ee7
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/070c2ee7
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/070c2ee7

Branch: refs/heads/master
Commit: 070c2ee7dd371157c1c242fe28b91f76ba47c5c5
Parents: 37bf54a
Author: Michael Stack 
Authored: Thu Dec 21 22:11:12 2017 -0800
Committer: Michael Stack 
Committed: Thu Dec 21 22:47:25 2017 -0800

--
 src/main/asciidoc/_chapters/ops_mgt.adoc | 136 ++
 1 file changed, 136 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/hbase/blob/070c2ee7/src/main/asciidoc/_chapters/ops_mgt.adoc
--
diff --git a/src/main/asciidoc/_chapters/ops_mgt.adoc 
b/src/main/asciidoc/_chapters/ops_mgt.adoc
index 7b0f89b..f5fa456 100644
--- a/src/main/asciidoc/_chapters/ops_mgt.adoc
+++ b/src/main/asciidoc/_chapters/ops_mgt.adoc
@@ -2704,4 +2704,140 @@ Viewing the Master log will give you insight on rsgroup 
operation.
 If it appears stuck, restart the Master process.
 
 
+[[normalizer]]
+== Region Normalizer
+
+The Region Normalizer tries to make Regions all in a table about the same in 
size.
+It does this by finding a rough average. Any region that is larger than twice 
this
+size is split. Any region that is much smaller is merged into an adjacent 
region.
+It is good to run the Normalizer on occasion on a down time after the cluster 
has
+been running a while or say after a burst of activity such as a large delete.
+
+(The bulk of the below detail was copied wholesale from the blog by Romil 
Choksi at
+link:https://community.hortonworks.com/articles/54987/hbase-region-normalizer.html[HBase
 Region Normalizer])
+
+The Region Normalizer is feature available since HBase-1.2. It runs a set of
+pre-calculated merge/split actions to resize regions that are either too
+large or too small compared to the average region size for a given table. 
Region
+Normalizer when invoked computes a normalization 'plan' for all of the tables 
in
+HBase. System tables (such as hbase:meta, hbase:namespace, Phoenix system 
tables
+etc) and user tables with normalization disabled are ignored while computing 
the
+plan. For normalization enabled tables, normalization plan is carried out in
+parallel across multiple tables.
+
+Normalizer can be enabled or disabled globally for the entire cluster using the
+‘normalizer_switch’ command in the HBase shell. Normalization can also be
+controlled on a per table basis, which is disabled by default when a table is
+created. Normalization for a table can be enabled or disabled by setting the
+NORMALIZATION_ENABLED table attribute to true or false.
+
+To check normalizer status and enable/disable normalizer
 
+[source,bash]
+
+hbase(main):001:0> normalizer_enabled
+true 
+0 row(s) in 0.4870 seconds
+ 
+hbase(main):002:0> normalizer_switch false
+true 
+0 row(s) in 0.0640 seconds
+ 
+hbase(main):003:0> normalizer_enabled
+false 
+0 row(s) in 0.0120 seconds
+ 
+hbase(main):004:0> normalizer_switch true
+false
+0 row(s) in 0.0200 seconds
+ 
+hbase(main):005:0> normalizer_enabled
+true
+0 row(s) in 0.0090 seconds
+
+
+When enabled, Normalizer is invoked in the background every 5 mins (by 
default),
+which can be configured using `hbase.normalization.period` in `hbase-site.xml`.
+Normalizer can also be invoked manually/programmatically at will using HBase 
shell’s
+`normalize` command. HBase by default uses `SimpleRegionNormalizer`, but users 
can
+design their own normalizer as long as they implement the RegionNormalizer 
Interface.
+Details about the logic used by `SimpleRegionNormalizer` to compute its 
normalization
+plan can be found 
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.html[here].
+
+The below example shows a normalization plan being computed for an user table, 
and
+merge action being taken as a result of the normalization plan computed by 
SimpleRegionNormalizer.
+
+Consider an user table with some pre-split regions having 3 equally large 
regions
+(about 100K rows) and 1 relatively small region (about 25K rows). Following is 
the
+snippet from an hbase meta table scan showing each of the pre-split regions 
for 
+the user table.
+
+
+table_p8ddpd6q5z,,1469494305548.68b9892220865cb6048 column=info:regioninfo, 
timestamp=1469494306375, value={ENCODED => 68b9892220865cb604809c950d1adf48, 
NAME => 'table_p8ddpd6q5z,,1469494305548.68b989222 09c950d1adf48.   
0865cb604809c950d1adf48.', STARTKEY => '', ENDKEY => '1'} 
+