abdullah alamoudi has posted comments on this change. Change subject: [NO ISSUE] Report all BufferCache write failures. ......................................................................
Patch Set 5: (6 comments) https://asterix-gerrit.ics.uci.edu/#/c/2787/5/hyracks-fullstack/hyracks/hyracks-storage-am-bloomfilter/src/main/java/org/apache/hyracks/storage/am/bloomfilter/impls/BloomFilter.java File hyracks-fullstack/hyracks/hyracks-storage-am-bloomfilter/src/main/java/org/apache/hyracks/storage/am/bloomfilter/impls/BloomFilter.java: PS5, Line 288: private volatile Throwable failure = null; > how can 2 threads access this? the thread doing the bulkloading and the Buffer cache writer thread. https://asterix-gerrit.ics.uci.edu/#/c/2787/5/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTree.java File hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTree.java: PS5, Line 1195: @Override : public void writeFailed(ICachedPage page, Throwable failure) { : if (this.failure == null) { : this.failure = failure; : } : } : : @Override : public boolean hasFailed() { : return failure != null; : } : : @Override : public Throwable getFailure() { : return failure; : } > Is it possible to extract a common abstract bulkloader class for this purpo probably possible.. let me see what can be done. https://asterix-gerrit.ics.uci.edu/#/c/2787/5/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/IndexWithBuddyBulkLoader.java File hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/IndexWithBuddyBulkLoader.java: PS5, Line 85: getFailure > What if both bulkloaders have failed? This method should only return any fa This would return the first it sees. The other one would appear in the logs only :-\ I don't think this is so bad. Is it? https://asterix-gerrit.ics.uci.edu/#/c/2787/5/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndexDiskComponent.java File hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndexDiskComponent.java: PS5, Line 116: if (callback.hasFailed()) { : throw HyracksDataException.create(callback.getFailure()); : } > why not else?.. The failure could be set in the call at line 114 https://asterix-gerrit.ics.uci.edu/#/c/2787/5/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/AsyncFIFOPageQueueManager.java File hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/AsyncFIFOPageQueueManager.java: PS5, Line 72: LOGGER.error("An attempt to write a page found buffer cache closed"); : ExitUtil.halt(ExitUtil.EC_ABNORMAL_TERMINATION); : } : } catch (InterruptedException e) { : LOGGER.error("IO Operation interrupted", e); : ExitUtil.halt(ExitUtil.EC_ABNORMAL_TERMINATION); > this condition will probably happen if we ask for shutdown while a write is I don't think this happens in that scenario since part of the shutdown is to close dataset lifecycle manager and when we do that, we wait for all ongoing io operations. https://asterix-gerrit.ics.uci.edu/#/c/2787/5/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/IFIFOPageQueue.java File hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/IFIFOPageQueue.java: PS5, Line 18: // Isn't a queue FIFO by default? rename to IPageWriteQueue? > the fifo in this just comes from how we were calling it at the time, in con I will not rename it in this change but would personally prefer if it is renamed. -- To view, visit https://asterix-gerrit.ics.uci.edu/2787 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I97fd3dccff85dab84d644359be6f66b15ee708ef Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Luo Chen <[email protected]> Gerrit-Reviewer: Michael Blow <[email protected]> Gerrit-Reviewer: Murtadha Hubail <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]> Gerrit-Reviewer: abdullah alamoudi <[email protected]> Gerrit-HasComments: Yes
