[
https://issues.apache.org/jira/browse/ACCUMULO-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13574805#comment-13574805
]
Josh Elser commented on ACCUMULO-1052:
--------------------------------------
Merging minor compactions were indeed disabled (max files increased
drastically) due to the extremely long minor compaction when ingesting into a
table which already has decently sized rfiles. Thinking about this makes me
think we need a ticket to configure a maximum size of files in which to
actually perform merging minor compactions on. Meaning, when a small amount of
data is waiting to be minor compacted, and the smallest file which already
exists for that tablet is very large (smallest among many very large files),
merging minor compactions really suck.
> Minor compactions not finishing before master kills tabletserver can very
> large number of files per tablet
> ----------------------------------------------------------------------------------------------------------
>
> Key: ACCUMULO-1052
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1052
> Project: Accumulo
> Issue Type: Improvement
> Components: master, tserver
> Affects Versions: 1.4.2
> Environment: Large, write-heavy cluster
> Reporter: Josh Elser
> Assignee: Eric Newton
>
> On a cluster that is being saturated with heavy ingest, a tserver is observed
> attempting to perform a minor compaction for a tablet with multiple WALs.
> Because of this, commits to this tablet end up being held.
> After churning on the minc for some time, the master's hold-time limit for
> tservers is exceeded, however the minc didn't finish. The tserver is forcibly
> killed, the tablet is migrated, recovery occurs on the new tserver and the
> problem repeats.
> Some of the minor compactions must finish, as the number of files for that
> tablet continue to grow, but major compactions must not have time to finish
> since the number of files grow unbounded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
