[
https://issues.apache.org/jira/browse/HBASE-3015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kannan Muthukkaruppan updated HBASE-3015:
-----------------------------------------
Description:
On RS crash, master processes the RS's logs, splits them into per region log
files, and puts them in recovered.edits sub-directory of the region.
It may be the case the some of these files contain only old edits that have
already been flushed, and don't need to be reapplied again. However, in this
case the file is not deleted, and stays in recovered.edits for ever. This will
slow down every future "open" of this region, as the region will unnecessarily
spend time processing this file.
In HRegion.java:replaceRecoveredEditsIfAny(), the code below checks if the
file we just processed contain any edits that were applied, and in that case
flushes the memstore into which things were being recovered.
{code}
if (seqid > minSeqId) {
// Then we added some edits to memory. Flush and cleanup split edit files.
internalFlushcache(null, seqid);
for (Path file: files) {
if (!this.fs.delete(file, false)) {
LOG.error("Failed delete of " + file);
} else {
LOG.debug("Deleted recovered.edits file=" + file);
}
}
}
{code}
But it is not clear why the 'for' loop to clean up the recovered.edits file is
inside the if check.
was:
On RS crash, master processes the RS's logs, splits them into per region log
files, and puts them in recovered.edits sub-directory of the region.
It may be the case the some of these files contain only old edits (that have
already been flushed), and don't need to be reapplied again. However, in this
case the file is not deleted, and stays in recovered.edits for ever. This will
slow down every future "open" of this region, as the region will unnecessarily
spend time processing this file.
In HRegion.java:replaceRecoveredEditsIfAny(), the code below checks if the
file we just processed contain any edits that were applied, and in that case
flushes the memstore into which things were being recovered.
{code}
if (seqid > minSeqId) {
// Then we added some edits to memory. Flush and cleanup split edit files.
internalFlushcache(null, seqid);
for (Path file: files) {
if (!this.fs.delete(file, false)) {
LOG.error("Failed delete of " + file);
} else {
LOG.debug("Deleted recovered.edits file=" + file);
}
}
}
{code}
But it is not clear why the 'for' loop to clean up the recovered.edits file is
inside the if check.
> recovered.edits files not deleted if it only contain edits that have already
> been flushed; hurts perf for all future opens of the region
> ----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-3015
> URL: https://issues.apache.org/jira/browse/HBASE-3015
> Project: HBase
> Issue Type: Bug
> Reporter: Kannan Muthukkaruppan
>
> On RS crash, master processes the RS's logs, splits them into per region log
> files, and puts them in recovered.edits sub-directory of the region.
> It may be the case the some of these files contain only old edits that have
> already been flushed, and don't need to be reapplied again. However, in this
> case the file is not deleted, and stays in recovered.edits for ever. This
> will slow down every future "open" of this region, as the region will
> unnecessarily spend time processing this file.
> In HRegion.java:replaceRecoveredEditsIfAny(), the code below checks if the
> file we just processed contain any edits that were applied, and in that case
> flushes the memstore into which things were being recovered.
> {code}
> if (seqid > minSeqId) {
> // Then we added some edits to memory. Flush and cleanup split edit
> files.
> internalFlushcache(null, seqid);
> for (Path file: files) {
> if (!this.fs.delete(file, false)) {
> LOG.error("Failed delete of " + file);
> } else {
> LOG.debug("Deleted recovered.edits file=" + file);
> }
> }
> }
> {code}
> But it is not clear why the 'for' loop to clean up the recovered.edits file
> is inside the if check.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
