Luo Chen has posted comments on this change. Change subject: Avoid always merging old components in prefix policy ......................................................................
Patch Set 5: (1 comment) > (5 comments) https://asterix-gerrit.ics.uci.edu/#/c/1818/5/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/PrefixMergePolicy.java File hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/PrefixMergePolicy.java: PS5, Line 303: // otherwise, we can look ahead to see if adding more younger components : // could pass the ratio check. > It looks to me that the old algorithm is O(2*n) for the worst case. However I haven't done any profiling yet, but seems to me this is not a big problem for the following reasons. First, the new algorithm skips all too large disk components (just like the old algorithm). Then it checks the remaining disk components to merge. The key point is that the number of the remaining components is very small, basically similar to maxToleranceComponentCount (normally 5 to 10 maybe?). It has optimizations to stop early for most cases. I think it should be fine, given the actual number components we need to look at is often very small. -- To view, visit https://asterix-gerrit.ics.uci.edu/1818 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I464da3fed38cded0aee7b319a35664eae069a2ba Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Luo Chen <[email protected]> Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Jianfeng Jia <[email protected]> Gerrit-Reviewer: Luo Chen <[email protected]> Gerrit-Reviewer: Yingyi Bu <[email protected]> Gerrit-Reviewer: abdullah alamoudi <[email protected]> Gerrit-HasComments: Yes
