Repository: hbase Updated Branches: refs/heads/branch-1.1 434097db0 -> 540bf082a refs/heads/branch-1.2 31677c0aa -> 041fbe71b refs/heads/branch-1.3 17f11ae6c -> 565527c60 refs/heads/branch-1.4 846753c18 -> 9a075fe73 refs/heads/branch-2 9c85d0017 -> fb79e9d4a refs/heads/master df98d6848 -> d89682ea9
HBASE-19223 Remove references to Date Tiered compaction from branch-1.2 ref guide Signed-off-by: Yu Li <l...@apache.org> Signed-off-by: Mike Drob <md...@apache.org> Project: http://git-wip-us.apache.org/repos/asf/hbase/repo Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/041fbe71 Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/041fbe71 Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/041fbe71 Branch: refs/heads/branch-1.2 Commit: 041fbe71b6429170576395b35603e07157acc585 Parents: 31677c0 Author: Sean Busbey <bus...@apache.org> Authored: Thu Nov 9 08:04:20 2017 -0600 Committer: Sean Busbey <bus...@apache.org> Committed: Wed Nov 15 10:28:14 2017 -0600 ---------------------------------------------------------------------- src/main/asciidoc/_chapters/architecture.adoc | 101 --------------------- src/main/asciidoc/_chapters/upgrading.adoc | 11 ++- 2 files changed, 7 insertions(+), 105 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hbase/blob/041fbe71/src/main/asciidoc/_chapters/architecture.adoc ---------------------------------------------------------------------- diff --git a/src/main/asciidoc/_chapters/architecture.adoc b/src/main/asciidoc/_chapters/architecture.adoc index 9f59cd5..6ab5f48 100644 --- a/src/main/asciidoc/_chapters/architecture.adoc +++ b/src/main/asciidoc/_chapters/architecture.adoc @@ -2060,107 +2060,6 @@ Why? NOTE: This information is now included in the configuration parameter table in <<compaction.parameters>>. -[[ops.date.tiered]] -===== Date Tiered Compaction - -Date tiered compaction is a date-aware store file compaction strategy that is beneficial for time-range scans for time-series data. - -[[ops.date.tiered.when]] -===== When To Use Date Tiered Compactions - -Consider using Date Tiered Compaction for reads for limited time ranges, especially scans of recent data - -Don't use it for - -* random gets without a limited time range -* frequent deletes and updates -* Frequent out of order data writes creating long tails, especially writes with future timestamps -* frequent bulk loads with heavily overlapping time ranges - -.Performance Improvements -Performance testing has shown that the performance of time-range scans improve greatly for limited time ranges, especially scans of recent data. - -[[ops.date.tiered.enable]] -====== Enabling Date Tiered Compaction - -You can enable Date Tiered compaction for a table or a column family, by setting its `hbase.hstore.engine.class` to `org.apache.hadoop.hbase.regionserver.DateTieredStoreEngine`. - -You also need to set `hbase.hstore.blockingStoreFiles` to a high number, such as 60, if using all default settings, rather than the default value of 12). Use 1.5~2 x projected file count if changing the parameters, Projected file count = windows per tier x tier count + incoming window min + files older than max age - -You also need to set `hbase.hstore.compaction.max` to the same value as `hbase.hstore.blockingStoreFiles` to unblock major compaction. - -.Procedure: Enable Date Tiered Compaction -. Run one of following commands in the HBase shell. - Replace the table name `orders_table` with the name of your table. -+ -[source,sql] ----- -alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.DateTieredStoreEngine', 'hbase.hstore.blockingStoreFiles' => '60', 'hbase.hstore.compaction.min'=>'2', 'hbase.hstore.compaction.max'=>'60'} -alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.DateTieredStoreEngine', 'hbase.hstore.blockingStoreFiles' => '60', 'hbase.hstore.compaction.min'=>'2', 'hbase.hstore.compaction.max'=>'60'}} -create 'orders_table', 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.DateTieredStoreEngine', 'hbase.hstore.blockingStoreFiles' => '60', 'hbase.hstore.compaction.min'=>'2', 'hbase.hstore.compaction.max'=>'60'} ----- - -. Configure other options if needed. - See <<ops.date.tiered.config>> for more information. - -.Procedure: Disable Date Tiered Compaction -. Set the `hbase.hstore.engine.class` option to either nil or `org.apache.hadoop.hbase.regionserver.DefaultStoreEngine`. - Either option has the same effect. - Make sure you set the other options you changed to the original settings too. -+ -[source,sql] ----- -alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.DefaultStoreEngine'ï¼ 'hbase.hstore.blockingStoreFiles' => '12', 'hbase.hstore.compaction.min'=>'6', 'hbase.hstore.compaction.max'=>'12'}} ----- - -When you change the store engine either way, a major compaction will likely be performed on most regions. -This is not necessary on new tables. - -[[ops.date.tiered.config]] -====== Configuring Date Tiered Compaction - -Each of the settings for date tiered compaction should be configured at the table or column family level. -If you use HBase shell, the general command pattern is as follows: - -[source,sql] ----- -alter 'orders_table', CONFIGURATION => {'key' => 'value', ..., 'key' => 'value'}} ----- - -[[ops.date.tiered.config.parameters]] -.Tier Parameters - -You can configure your date tiers by changing the settings for the following parameters: - -.Date Tier Parameters -[cols="1,1a", frame="all", options="header"] -|=== -| Setting -| Notes - -|`hbase.hstore.compaction.date.tiered.max.storefile.age.millis` -|Files with max-timestamp smaller than this will no longer be compacted.Default at Long.MAX_VALUE. - -| `hbase.hstore.compaction.date.tiered.base.window.millis` -| Base window size in milliseconds. Default at 6 hours. - -| `hbase.hstore.compaction.date.tiered.windows.per.tier` -| Number of windows per tier. Default at 4. - -| `hbase.hstore.compaction.date.tiered.incoming.window.min` -| Minimal number of files to compact in the incoming window. Set it to expected number of files in the window to avoid wasteful compaction. Default at 6. - -| `hbase.hstore.compaction.date.tiered.window.policy.class` -| The policy to select store files within the same time window. It doesnât apply to the incoming window. Default at exploring compaction. This is to avoid wasteful compaction. -|=== - -[[ops.date.tiered.config.compaction.throttler]] -.Compaction Throttler - -With tiered compaction all servers in the cluster will promote windows to higher tier at the same time, so using a compaction throttle is recommended: -Set `hbase.regionserver.throughput.controller` to `org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController`. - -NOTE: For more information about date tiered compaction, please refer to the design specification at https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8 [[ops.stripe]] ===== Experimental: Stripe Compactions http://git-wip-us.apache.org/repos/asf/hbase/blob/041fbe71/src/main/asciidoc/_chapters/upgrading.adoc ---------------------------------------------------------------------- diff --git a/src/main/asciidoc/_chapters/upgrading.adoc b/src/main/asciidoc/_chapters/upgrading.adoc index d731542..3acb36e 100644 --- a/src/main/asciidoc/_chapters/upgrading.adoc +++ b/src/main/asciidoc/_chapters/upgrading.adoc @@ -176,16 +176,16 @@ In the minor version-particular sections below, we call out where the versions a == Upgrade Paths [[upgrade1.0]] -=== Upgrading from 0.98.x to 1.0.x +=== Upgrading from 0.98.x to 1.x -In this section we first note the significant changes that come in with 1.0.0 HBase and then we go over the upgrade process. Be sure to read the significant changes section with care so you avoid surprises. +In this section we first note the significant changes that come in with 1.0.0+ HBase and then we go over the upgrade process. Be sure to read the significant changes section with care so you avoid surprises. ==== Changes of Note! -In here we list important changes that are in 1.0.0 since 0.98.x., changes you should be aware that will go into effect once you upgrade. +In here we list important changes that are in 1.0.0+ since 0.98.x., changes you should be aware that will go into effect once you upgrade. [[zookeeper.3.4]] -.ZooKeeper 3.4 is required in HBase 1.0.0 +.ZooKeeper 3.4 is required in HBase 1.0.0+ See <<zookeeper.requirements>>. [[default.ports.changed]] @@ -218,6 +218,9 @@ to miss data. In particular, 0.98.11 defaults `hbase.client.scanner.max.result.s to 2 MB but other versions default to larger values. For this reason, be very careful using 0.98.11 servers with any other client version. +.Availability of Date Tiered Compaction. +The Date Tiered Compaction feature available as of 0.98.19 is available in the 1.y release line starting in release 1.3.0. If you have enabled this feature for any tables you must upgrade to version 1.3.0 or later. If you attempt to use an earlier 1.y release, any tables configured to use date tiered compaction will fail to have their regions open. + [[upgrade1.0.rolling.upgrade]] ==== Rolling upgrade from 0.98.x to HBase 1.0.0 .From 0.96.x to 1.0.0
