Major compactions from shell may not major compact all families
---------------------------------------------------------------
Key: HBASE-2770
URL: https://issues.apache.org/jira/browse/HBASE-2770
Project: HBase
Issue Type: Bug
Affects Versions: 0.20.4
Reporter: Dave Latham
Fix For: 0.20.6
As part of a data center migration, I initiated a major_compaction request on
all tables from the shell. A few hours later, all the region servers in the
cluster appeared to have completed the compactions and all compactionQueue
metrics were back to 0. However, some column families of some regions had not
actually done a major compaction.
Digging through logs and code, it looks like the following happened. The shell
makes a major compaction request which sets HRegion.forceMajorCompaction to
true for every region. Periodically, the HRegionServer.MajorCompactionChecker
checks to see if a major compaction is needed in any family's store. If so,
calls CompactSplitThread.compactionRequested which ends up setting the region
forceMajorCompaction to false, even if it is already in the compaction queue
and set to true. Then, when that region comes off the queue to be compacted,
each family/store separately checks for whether it should do a major
compaction, so some families may not do so.
(This is not good if, for example, you're doing a DistCp of the hbase dir and
later on the cluster decides to do a compaction on those files and deletes ones
the DistCp job is looking for, causing it to fail.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
