Chen Luo created ASTERIXDB-2600:
-----------------------------------
Summary: Introduce ConcurrentMergePolicy
Key: ASTERIXDB-2600
URL: https://issues.apache.org/jira/browse/ASTERIXDB-2600
Project: Apache AsterixDB
Issue Type: Improvement
Reporter: Chen Luo
Assignee: Chen Luo
The default PrefixMergePolicy has a number of problems: it only schedules 1
merge at a time; it stops merging components when they are too large. Both
problems will lead to undesirable performance behavior. This issue introduces a
ConcurrentMergePolicy that fixes these problems.
It has the following four parameters:
* minComponentMergeCount: the minimum number of components to trigger a merge
* maxComponentMergeCount: the maximum number of components per merge
* maxComponentCount: the maximum number of components tolerated in total; when
this number is reached, flush will be stopped
* sizeRatio: a merge is scheduled if the size of the oldest component <=
sizeRatio * total size of youngest components. A larger size ratio(>1) implies
fewer merges, but this leads to more components. A smaller size ratio (<1)
implies more merges, but this improves query performance and space utilization.
Concurrent merges will be scheduled as well. Given a sequence of components
ordered by newest to oldest, this policy first finds a longest prefix of this
sequence so that no component is being merged. The merge decision will then be
based on this prefix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
