Luo Chen has posted comments on this change. Change subject: Avoid always merging old components in prefix policy ......................................................................
Patch Set 7: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/1818/6/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/PrefixMergePolicy.java File hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/PrefixMergePolicy.java: PS6, Line 273: } > What I proposed is almost the same as what you're doing. Since the current I see. But I suspect that this new change wouldn't bring too much improvement to the current patch, but instead making it more complicated (right now the logic is already a bit complicated...) Still, I think this idea is a specialization of the level-based policy, where all flushed disk components have level 0, but merged disk components have level 1 (here we don't differentiate between multiple rounds of merges). Moreover, if consider the layout of the disk components resulting from this policy, then the actual checking wound't takes too much time. Suppose MaxMergableSize = 100M, MaxToleranceCount = 5, initial flushed disk component has size 1M, then the "worst" layout of 100 disk components would be like: >100M, >100M , ..., >100M, 61M, 25M, 11M, 5M, 1M, 1M, 1M, 1M Here the prefix policy wound't schedule any merges. But when a new disk component is added, the policy would first skip all too large components, and only consider the newest (smallest) components. The size the mergeable components considered by the policy is always small, and it actually has some upper bound. -- To view, visit https://asterix-gerrit.ics.uci.edu/1818 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I464da3fed38cded0aee7b319a35664eae069a2ba Gerrit-PatchSet: 7 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Luo Chen <[email protected]> Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Jianfeng Jia <[email protected]> Gerrit-Reviewer: Luo Chen <[email protected]> Gerrit-Reviewer: Yingyi Bu <[email protected]> Gerrit-Reviewer: abdullah alamoudi <[email protected]> Gerrit-HasComments: Yes
