[
https://issues.apache.org/jira/browse/HBASE-12657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240081#comment-14240081
]
Vladimir Rodionov commented on HBASE-12657:
-------------------------------------------
There is a difference between 0.94 and 0.98+ in how we remove excess files:
0.94 - from the head
{code}
if(majorcompaction) {
if (compactSelection.getFilesToCompact().size() >
this.maxFilesToCompact) {
LOG.debug("Warning, compacting more than " + this.maxFilesToCompact +
" files, probably because of a user-requested major compaction");
if(priority != PRIORITY_USER) {
LOG.error("Compacting more than max files on a non user-requested
compaction");
}
}
} else if (compactSelection.getFilesToCompact().size() >
this.maxFilesToCompact) {
// all files included in this compaction, up to max
int pastMax = compactSelection.getFilesToCompact().size() -
this.maxFilesToCompact;
compactSelection.getFilesToCompact().subList(0, pastMax).clear();
}
{code}
trunk & 0.98 - from the tail
{code}
if (isMajorCompaction && isUserCompaction) {
LOG.debug("Warning, compacting more than " +
comConf.getMaxFilesToCompact() +
" files because of a user-requested major compaction");
} else {
LOG.debug("Too many admissible files. Excluding " + excess
+ " files from compaction candidates");
candidates.subList(comConf.getMaxFilesToCompact(),
candidates.size()).clear();
}
{code}
> The Region is not being split and far exceeds the desired maximum size.
> -----------------------------------------------------------------------
>
> Key: HBASE-12657
> URL: https://issues.apache.org/jira/browse/HBASE-12657
> Project: HBase
> Issue Type: Bug
> Components: Compaction
> Affects Versions: 0.98.8, 0.94.25, 0.99.2
> Reporter: Vladimir Rodionov
> Assignee: Vladimir Rodionov
> Fix For: 1.0.0, 2.0.0, 0.94.26, 0.98.9
>
>
> We are seeing this behavior when creating indexes in one of our environment.
> When an index is being created, most of the "requests" go into a single
> region. The amount of time to create an index seems to take longer than
> usual and it can take days for the regions to compact and split after the
> index is created.
> Here is a du of the HBase index table:
> {code}
> -bash-4.1$ sudo -su hdfs hadoop fs -du /hbase/43681
> 705 /hbase/43681/.tableinfo.0000000001
> 0 /hbase/43681/.tmp
> 27981697293 /hbase/43681/0492e22092e21d35fca8e779b21ec797
> 539687093 /hbase/43681/832298c4e975fc47210feb6bac3d2f71
> 560660531 /hbase/43681/be9bdb3bdf9365afe5fe90db4247d82c
> 7081938297 /hbase/43681/cd440e524f96fbe0719b2fe969848560
> 6297860287 /hbase/43681/dc893a2d8daa08c689dc69e6bb2c5b50
> 7189607722 /hbase/43681/ffbceaea5e2f142dbe6cd4cbeacc00e8
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
