HBASE-18504 Add documentation for WAL compression Signed-off-by: Michael Stack <st...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/hbase/repo Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/63e313b5 Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/63e313b5 Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/63e313b5 Branch: refs/heads/HBASE-14070.HLC Commit: 63e313b5c0d7c56d9cf9602e3c204214331189d5 Parents: 70c4f78 Author: Peter Somogyi <psomo...@cloudera.com> Authored: Wed Aug 2 17:00:52 2017 +0200 Committer: Michael Stack <st...@apache.org> Committed: Tue Aug 15 12:54:08 2017 -0700 ---------------------------------------------------------------------- src/main/asciidoc/_chapters/architecture.adoc | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hbase/blob/63e313b5/src/main/asciidoc/_chapters/architecture.adoc ---------------------------------------------------------------------- diff --git a/src/main/asciidoc/_chapters/architecture.adoc b/src/main/asciidoc/_chapters/architecture.adoc index ebb0677..2ded813 100644 --- a/src/main/asciidoc/_chapters/architecture.adoc +++ b/src/main/asciidoc/_chapters/architecture.adoc @@ -1216,6 +1216,22 @@ This will be the default for HBase 0.99 (link:https://issues.apache.org/jira/bro You must also enable HFile version 3 (which is the default HFile format starting in HBase 0.99. See link:https://issues.apache.org/jira/browse/HBASE-10855[HBASE-10855]). Distributed log replay is unsafe for rolling upgrades. +[[wal.compression]] +==== WAL Compression ==== + +The content of the WAL can be compressed using LRU Dictionary compression. +This can be used to speed up WAL replication to different datanodes. +The dictionary can store up to 2^15^ elements; eviction starts after this number is exceeded. + +To enable WAL compression, set the `hbase.regionserver.wal.enablecompression` property to `true`. +The default value for this property is `false`. +By default, WAL tag compression is turned on when WAL compression is enabled. +You can turn off WAL tag compression by setting the `hbase.regionserver.wal.tags.enablecompression` property to 'false'. + +A possible downside to WAL compression is that we lose more data from the last block in the WAL if it ill-terminated +mid-write. If entries in this last block were added with new dictionary entries but we failed persist the amended +dictionary because of an abrupt termination, a read of this last block may not be able to resolve last-written entries. + [[wal.disable]] ==== Disabling the WAL