[
https://issues.apache.org/jira/browse/ACCUMULO-4692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114653#comment-16114653
]
Adam Fuchs commented on ACCUMULO-4692:
--------------------------------------
Interesting -- since I was always compacting a range with the end equalling the
tablet end row I wasn't thinking of the challenge of finding the next split
point. However, I can think of a couple of ways of getting a tighter end in the
scan:
(1) Keep the end row used to launch compactions in the previous REPO or the end
row of interest from the last time the CompactionDriver.isReady method was
called in this FATE transaction. Detect when it's wrong (i.e. after a tablet
merge) and fall back to a bigger scan.
(2) Use an iterator to detect when the tablet being scanned has a prev end row
after the end of the scanned range and terminate the scan.
Found this on a version derived from 1.7.2.
> CompactionDriver leaves abandoned metadata scans
> ------------------------------------------------
>
> Key: ACCUMULO-4692
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4692
> Project: Accumulo
> Issue Type: Bug
> Components: fate
> Reporter: Adam Fuchs
>
> We wrote a tool to kick off tablet compactions in the background while
> minimizing compaction load per-server. The tool uses range compaction on one
> tablet per call. We're seeing a high number of scans on the metadata table
> (~7,000 on a ~100 node cluster).
> The metadata query in the isReady() method of CompactionDriver that is used
> to see if the compaction has completed uses a range that goes to the end of
> the metadata entries for the given table, but it stops consuming the results
> of the scanner at the end of the compaction range. isReady gets called in a
> pretty tight loop, especially with hundreds of compactions running
> concurrently. Seems like we should limit the scan to the metadata range
> associated with the compaction so that the scan can get cleaned up quickly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
