VladRodionov commented on a change in pull request #1232: HBASE-23198 Update ref guide for distributed MOB compaction. URL: https://github.com/apache/hbase/pull/1232#discussion_r387231459
########## File path: src/main/asciidoc/_chapters/hbase_mob.adoc ########## @@ -79,63 +72,209 @@ hcd.setMobThreshold(102400L); ---- ==== -=== Configure MOB Compaction Policy +=== Testing MOB + +The utility `org.apache.hadoop.hbase.IntegrationTestIngestWithMOB` is provided to assist with testing +the MOB feature. The utility is run as follows: +[source,bash] +---- +$ sudo -u hbase hbase org.apache.hadoop.hbase.IntegrationTestIngestWithMOB \ + -threshold 1024 \ + -minMobDataSize 512 \ + -maxMobDataSize 5120 +---- + +* `*threshold*` is the threshold at which cells are considered to be MOBs. + The default is 1 kB, expressed in bytes. +* `*minMobDataSize*` is the minimum value for the size of MOB data. + The default is 512 B, expressed in bytes. +* `*maxMobDataSize*` is the maximum value for the size of MOB data. + The default is 5 kB, expressed in bytes. + +=== MOB architecture + +==== Overview +This section is derived from information found in +link:https://issues.apache.org/jira/browse/HBASE-11339[HBASE-11339]. For more information see +the last version of the design doc created during that work: +"link:https://github.com/apache/hbase/blob/master/dev-support/design-docs/HBASE-11339%20MOB%20GA%20design.pdf[HBASE-11339 MOB GA design.pdf]". + +The MOB feature reduces the overall IO load for configured column families by storing values that +are larger than the configured threshold outside of the normal regions to avoid splits, merges, and +most importantly normal compactions. + +When a cell is first written to a region it is stored in the WAL and memstore regardless of value +size. When memstores from a column family configured to use MOB are eventually flushed two hfiles +are written simultaneously. Cells with a value smaller than the threshold size are written to a +normal region hfile. Cells with a value larger than the threshold are written into a special MOB +hfile and also have a MOB reference cell written into the normal region HFile. As the Region Server +flushes a MOB enabled memstore and closes a given normal region HFile it appends metadata that lists +each of the special MOB hfiles referenced by the cells within. + +MOB reference cells have the same key as the cell they are based on. The value of the reference cell +is made up of two pieces of metadata: the size of the actual value and the MOB hfile that contains +the original cell. In addition to any tags originally written to HBase, the reference cell prepends +two additional tags. The first is a marker tag that says the cell is a MOB reference. This can be +used later to scan specifically just for reference cells. The second stores the namespace and table +at the time the MOB hfile is written out. This tag is used to optimize how the MOB system finds +the underlying value in MOB hfiles after a series of HBase snapshot operations (ref HBASE-12332). +Note that tags are only available within HBase servers and by default are not sent over RPCs. + +All MOB hfiles for a given table are managed within a logical region that does not directly serve +requests. When these MOB hfiles are created from a flush or MOB compaction they are placed in a +dedicated mob data area under the hbase root directory specific to the namespace, table, mob +logical region, and column family. In general that means a path structured like: -By default, MOB files for one specific day are compacted into one large MOB file. -To reduce MOB file count more, there are other MOB Compaction policies supported. +---- +%HBase Root Dir%/mobdir/data/%namespace%/%table%/%logical region%/%column family%/ +---- -daily policy - compact MOB Files for one day into one large MOB file (default policy) -weekly policy - compact MOB Files for one week into one large MOB file -montly policy - compact MOB Files for one month into one large MOB File +With default configs, an example table named 'some_table' in the +default namespace with a MOB enabled column family named 'foo' this HDFS directory would be -.Configure MOB compaction policy Using HBase Shell ---- -hbase> create 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 102400, MOB_COMPACT_PARTITION_POLICY => 'daily'} -hbase> create 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 102400, MOB_COMPACT_PARTITION_POLICY => 'weekly'} -hbase> create 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 102400, MOB_COMPACT_PARTITION_POLICY => 'monthly'} +/hbase/mobdir/data/default/some_table/372c1b27e3dc0b56c3a031926e5efbe9/foo/ +---- + +These MOB hfiles are maintained by special chores in the HBase Master and across the individual +Region Servers. Specifically those chores take care of enforcing TTLs and compacting them. Note that +this compaction is primarily a matter of controlling the total number of files in HDFS because our +operational assumptions for MOB data is that it will seldom update or delete. + +When a given MOB hfile is no longer needed as a result of our compaction process then a chore in +the Master will take care of moving it to the archive just +like any normal hfile. Because the table's mob region is independent of all the normal regions it +can coexist with them in the regular archive storage area: -hbase> alter 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 102400, MOB_COMPACT_PARTITION_POLICY => 'daily'} -hbase> alter 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 102400, MOB_COMPACT_PARTITION_POLICY => 'weekly'} -hbase> alter 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 102400, MOB_COMPACT_PARTITION_POLICY => 'monthly'} +---- +/hbase/archive/data/default/some_table/372c1b27e3dc0b56c3a031926e5efbe9/foo/ ---- -=== Configure MOB Compaction mergeable threshold +The same hfile cleaning chores that take care of eventually deleting unneeded archived files from +normal regions thus also will take care of these MOB hfiles. -If the size of a mob file is less than this value, it's regarded as a small file and needs to -be merged in mob compaction. The default value is 1280MB. +==== MOB compaction + +Each time the memstore for a MOB enabled column family performs a flush HBase will write values over +the MOB threshold into MOB specific hfiles. When normal region compaction occurs the Region Server +rewrites the normal data files while maintaining references to these MOB files without rewriting +them. Normal client lookups for MOB values transparently will receive the original values because +the Region Server internals take care of using the reference data to then pull the value out of a +specific MOB file. This indirection means that building up a large number of MOB hfiles doesn't +impact the overall time to retrieve any specific MOB cell. Thus, we need not perform compactions of +the MOB hfiles nearly as often as normal hfiles. As a result, HBase saves IO by not rewriting MOB +hfiles as a part of the periodic compactions a Region Server does on its own. + +However, if deletes and updates of MOB cells are frequent then this indirection will begin to waste +space. The only way to stop using the space of a particular MOB hfile is to ensure no cells still +hold references to it. To do that we need to ensure we have written the current values into a new +MOB hfile. If our backing filesystem has a limitation on the number of files that can be present, as +HDFS does, then even if we do not have deletes or updates of MOB cells eventually there will be a +sufficient number of MOB hfiles that we will need to coallesce them. + +Periodically a chore in the master coordinates having the region servers +perform a special major compaction that also handles rewritting new MOB files. Because this +rewriting has the advantage of looking across all active cells for the region our several small MOB +files should end up as a single MOB file per region. The chore defaults to running weekly and can be +configured by setting `hbase.mob.compaction.chore.period` to the desired period in seconds. ==== [source,xml] ---- <property> - <name>hbase.mob.compaction.mergeable.threshold</name> - <value>10000000000</value> + <name>hbase.mob.compaction.chore.period</name> + <value>2592000</value> + <description>Example of changing the chore period from a week to a month.</description> </property> ---- ==== -=== Testing MOB +By default, the periodic MOB compaction coordination chore will attempt to keep every region +busy doing compactions in parallel in order to maximize the amount of work done on the cluster. +If you need to tune the amount of IO this compaction generates on the underlying filesystem, you +can control how many concurrent region-level compaction requests are allowed by setting +`hbase.mob.major.compaction.region.batch.size` to an integer number greater than zero. Review comment: 0 - means no limits, should be mentioned. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
