Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
abdullah alamoudi has submitted this change and it was merged.
Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources
..
[NO ISSUE][STO] Skip flush recovery of empty resources
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
- Before this change, recovery would throw a NullPointerException
on recovery of a flush operation on a component without
update logs.
- Since this can happen, we simply check for the case and skip the
flush.
Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2844
Tested-by: Jenkins
Contrib: Jenkins
Reviewed-by: abdullah alamoudi
Integration-Tests: Jenkins
Reviewed-by: Murtadha Hubail
---
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
1 file changed, 18 insertions(+), 12 deletions(-)
Approvals:
Anon. E. Moose #1000171:
abdullah alamoudi: Looks good to me, but someone else must approve
Jenkins: Verified; ; Verified
Murtadha Hubail: Looks good to me, approved
Objections:
Jenkins: Violations found
diff --git
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
index 5e8a5e8..adf9960 100644
---
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
+++
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
@@ -296,7 +296,7 @@
((INcApplicationContext)
(serviceCtx.getApplicationContext())).getIndexCheckpointManagerProvider();
Map resourcesMap =
localResourceRepository.loadAndGetAllResources();
-Map resourceId2MaxLSNMap = new HashMap<>();
+final Map resourceId2MaxLSNMap = new HashMap<>();
TxnEntityId tempKeyTxnEntityId = new TxnEntityId(-1, -1, -1, null, -1,
false);
ILogRecord logRecord = null;
@@ -399,19 +399,25 @@
// we only need to flush open indexes here (opened
by previous update records)
// if an index has no ongoing updates, then it's
memory component must be empty
// and there is nothing to flush
-for (IndexInfo iInfo :
dsInfo.getIndexes().values()) {
+for (final IndexInfo iInfo :
dsInfo.getIndexes().values()) {
if (iInfo.isOpen() && iInfo.getPartition() ==
partition) {
-maxDiskLastLsn =
resourceId2MaxLSNMap.get(iInfo.getResourceId());
-index = iInfo.getIndex();
-if (logRecord.getLSN() > maxDiskLastLsn
-&&
!index.isCurrentMutableComponentEmpty()) {
-// schedule flush
-redoFlush(index, logRecord);
-redoCount++;
+Long maxLsnBeforeFlush =
resourceId2MaxLSNMap.get(iInfo.getResourceId());
+if (maxLsnBeforeFlush != null) {
+// If there was at least one update to
the resource.
+// IMPORTANT: Don't remove the check
above
+// This check is to support indexes
without transaction logs
+maxDiskLastLsn = maxLsnBeforeFlush;
+index = iInfo.getIndex();
+if (logRecord.getLSN() > maxDiskLastLsn
+&&
!index.isCurrentMutableComponentEmpty()) {
+// schedule flush
+redoFlush(index, logRecord);
+redoCount++;
+} else {
+// TODO: update checkpoint file?
+}
} else {
-// otherwise, do nothing since this
component had no records when flush was
-// scheduled.. TODO: update checkpoint
file? and do the
-// lsn checks from the checkpoint file
+// TODO: update checkpoint file?
}
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/2844
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcd
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Murtadha Hubail has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Luo Chen Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Anon. E. Moose #1000171 has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Contrib+1 Analytics Compatibility Tests Successful https://goo.gl/S8bVFc : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Luo Chen Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Anon. E. Moose #1000171 has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Contrib+1 Analytics Compatibility Tests Successful https://goo.gl/Aic3EX : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Luo Chen Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/7044/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Luo Chen Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
abdullah alamoudi has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Luo Chen Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Contrib+1 BAD Compatibility Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/3518/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Luo Chen Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/3518/ -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Luo Chen Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/4586/ (13/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Luo Chen Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Anon. E. Moose #1000171 has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Analytics Compatibility Compilation Successful https://goo.gl/54b61o : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Luo Chen Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/4616/ (12/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Luo Chen Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Anon. E. Moose #1000171 has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Analytics Compatibility Compilation Successful https://goo.gl/Gt7u3b : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Luo Chen Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/7044/ -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Luo Chen Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/4413/ (6/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/9666/ (9/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-format/4156/ (2/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage-jre10/475/ (11/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-ensure-ancestor/2203/ (8/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/hyracks-gerrit/4103/ (7/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-java10/433/ (3/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/4191/ (5/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/8129/ (10/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage/4745/ (4/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/4195/ (1/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/4615/ (12/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
abdullah alamoudi has uploaded a new patch set (#2). Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. [NO ISSUE][STO] Skip flush recovery of empty resources - user model changes: no - storage format changes: no - interface changes: no Details: - Before this change, recovery would throw a NullPointerException on recovery of a flush operation on a component without update logs. - Since this can happen, we simply check for the case and skip the flush. Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee --- M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java 1 file changed, 18 insertions(+), 12 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/44/2844/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/4190/ (7/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/9665/ (1/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/4412/ (5/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/hyracks-gerrit/4102/ (6/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
abdullah alamoudi has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/2844
Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources
..
[NO ISSUE][STO] Skip flush recovery of empty resources
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
- Before this change, recovery would throw a NullPointerException
on recovery of a flush operation on a component without
update logs.
- Since this can happen, we simply check for the case and skip the
flush.
Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee
---
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
1 file changed, 19 insertions(+), 13 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/44/2844/1
diff --git
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
index 5e8a5e8..ee75be1 100644
---
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
+++
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
@@ -296,7 +296,7 @@
((INcApplicationContext)
(serviceCtx.getApplicationContext())).getIndexCheckpointManagerProvider();
Map resourcesMap =
localResourceRepository.loadAndGetAllResources();
-Map resourceId2MaxLSNMap = new HashMap<>();
+final Map resourceId2MaxLSNMap = new HashMap<>();
TxnEntityId tempKeyTxnEntityId = new TxnEntityId(-1, -1, -1, null, -1,
false);
ILogRecord logRecord = null;
@@ -399,19 +399,25 @@
// we only need to flush open indexes here (opened
by previous update records)
// if an index has no ongoing updates, then it's
memory component must be empty
// and there is nothing to flush
-for (IndexInfo iInfo :
dsInfo.getIndexes().values()) {
+for (final IndexInfo iInfo :
dsInfo.getIndexes().values()) {
if (iInfo.isOpen() && iInfo.getPartition() ==
partition) {
-maxDiskLastLsn =
resourceId2MaxLSNMap.get(iInfo.getResourceId());
-index = iInfo.getIndex();
-if (logRecord.getLSN() > maxDiskLastLsn
-&&
!index.isCurrentMutableComponentEmpty()) {
-// schedule flush
-redoFlush(index, logRecord);
-redoCount++;
-} else {
-// otherwise, do nothing since this
component had no records when flush was
-// scheduled.. TODO: update checkpoint
file? and do the
-// lsn checks from the checkpoint file
+Long maxLsnBeforeFlush =
resourceId2MaxLSNMap.get(iInfo.getResourceId());
+if (maxLsnBeforeFlush != null) {
+// If there was at least one update to
the resource.
+// IMPORTANT: Don't remove the check
above
+// This check is to support indexes
without transaction logs
+maxDiskLastLsn = maxLsnBeforeFlush;
+index = iInfo.getIndex();
+if (logRecord.getLSN() > maxDiskLastLsn
+&&
!index.isCurrentMutableComponentEmpty()) {
+// schedule flush
+redoFlush(index, logRecord);
+redoCount++;
+} else {
+// otherwise, do nothing since
this component had no records when flush was
+// scheduled.. TODO: update
checkpoint file? and do the
+// lsn checks from the checkpoint
file
+}
}
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/2844
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage/4744/ (3/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/4194/ (2/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-java10/432/ (9/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-ensure-ancestor/2202/ (8/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage-jre10/474/ (11/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/8128/ (10/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][STO] Skip flush recovery of empty resources
Jenkins has posted comments on this change. Change subject: [NO ISSUE][STO] Skip flush recovery of empty resources .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-format/4155/ (4/13) -- To view, visit https://asterix-gerrit.ics.uci.edu/2844 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib01d7513f43830109632760860d34ca3dcddeaee Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
