abdullah alamoudi has submitted this change and it was merged. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources ......................................................................
[NO ISSUE][STO] Skip flush recovery of empty resources - user model changes: no - storage format changes: no - interface changes: no Details: - Before this change, recovery would throw a NullPointerException on recovery of a flush operation on a component without update logs. - Since this can happen, we simply check for the case and skip the flush. Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Reviewed-on: https://asterix-gerrit.ics.uci.edu/2844 Tested-by: Jenkins <[email protected]> Contrib: Jenkins <[email protected]> Reviewed-by: abdullah alamoudi <[email protected]> Integration-Tests: Jenkins <[email protected]> Reviewed-by: Murtadha Hubail <[email protected]> --- M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java 1 file changed, 18 insertions(+), 12 deletions(-) Approvals: Anon. E. Moose #1000171: abdullah alamoudi: Looks good to me, but someone else must approve Jenkins: Verified; ; Verified Murtadha Hubail: Looks good to me, approved Objections: Jenkins: Violations found diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java index 5e8a5e8..adf9960 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java @@ -296,7 +296,7 @@ ((INcApplicationContext) (serviceCtx.getApplicationContext())).getIndexCheckpointManagerProvider(); Map<Long, LocalResource> resourcesMap = localResourceRepository.loadAndGetAllResources(); - Map<Long, Long> resourceId2MaxLSNMap = new HashMap<>(); + final Map<Long, Long> resourceId2MaxLSNMap = new HashMap<>(); TxnEntityId tempKeyTxnEntityId = new TxnEntityId(-1, -1, -1, null, -1, false); ILogRecord logRecord = null; @@ -399,19 +399,25 @@ // we only need to flush open indexes here (opened by previous update records) // if an index has no ongoing updates, then it's memory component must be empty // and there is nothing to flush - for (IndexInfo iInfo : dsInfo.getIndexes().values()) { + for (final IndexInfo iInfo : dsInfo.getIndexes().values()) { if (iInfo.isOpen() && iInfo.getPartition() == partition) { - maxDiskLastLsn = resourceId2MaxLSNMap.get(iInfo.getResourceId()); - index = iInfo.getIndex(); - if (logRecord.getLSN() > maxDiskLastLsn - && !index.isCurrentMutableComponentEmpty()) { - // schedule flush - redoFlush(index, logRecord); - redoCount++; + Long maxLsnBeforeFlush = resourceId2MaxLSNMap.get(iInfo.getResourceId()); + if (maxLsnBeforeFlush != null) { + // If there was at least one update to the resource. + // IMPORTANT: Don't remove the check above + // This check is to support indexes without transaction logs + maxDiskLastLsn = maxLsnBeforeFlush; + index = iInfo.getIndex(); + if (logRecord.getLSN() > maxDiskLastLsn + && !index.isCurrentMutableComponentEmpty()) { + // schedule flush + redoFlush(index, logRecord); + redoCount++; + } else { + // TODO: update checkpoint file? + } } else { - // otherwise, do nothing since this component had no records when flush was - // scheduled.. TODO: update checkpoint file? and do the - // lsn checks from the checkpoint file + // TODO: update checkpoint file? } } } -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin <[email protected]> Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Luo Chen <[email protected]> Gerrit-Reviewer: Michael Blow <[email protected]> Gerrit-Reviewer: Murtadha Hubail <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]> Gerrit-Reviewer: abdullah alamoudi <[email protected]>
