Luo Chen has posted comments on this change. Change subject: Implemented Disk Components Alignment Based on IDs ......................................................................
Patch Set 6: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/1761/6/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/CorrelatedPrefixMergePolicy.java File asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/CorrelatedPrefixMergePolicy.java: PS6, Line 103: isMergeLagging( > I am strongly opposed to delay of flushes for the sake of merges. Not sure Both solutions (with or without flow control) are solution OK to me. But without flow control, one consequence is that there could be more and more lagged disk components accumulated on the disk along with the data ingestion. But eventually when the ingestion stops, the disk components would be merged as expected. Any suggestions or comments from other reviewers? However, for the correlated merge policy, the merge operation shouldn't be triggered by any secondary index. The reason is that the secondary index disk component is usually much smaller than the primary index disk component. If the merge request is sent by the secondary index (for all indexes), then we would get much larger disk components for the primary, which breaks the criteria for the merge policy. For example, if we set the MaxMergableComponentSize=128M (this tells the merge policy that if any disk component is larger than this size, then never merge it again), when the primary index disk component reaches this limit, the corresponding secondary index disk component is only about 4-5M. -- To view, visit https://asterix-gerrit.ics.uci.edu/1761 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I768ee9ac0a8d3c99c631086093a6b778b2e7588e Gerrit-PatchSet: 6 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Luo Chen <[email protected]> Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Jianfeng Jia <[email protected]> Gerrit-Reviewer: Luo Chen <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]> Gerrit-Reviewer: Yingyi Bu <[email protected]> Gerrit-Reviewer: abdullah alamoudi <[email protected]> Gerrit-HasComments: Yes
