anoopsjohn commented on a change in pull request #939: HBASE-23349 : Notify
readers if archival of compacted store files is stuck due to scanners
URL: https://github.com/apache/hbase/pull/939#discussion_r358291116
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
##########
@@ -2723,6 +2719,53 @@ private void
removeCompactedfiles(Collection<HStoreFile> compactedfiles)
}
}
+ private boolean closeFileReaderForArchival(List<HStoreFile> filesToRemove,
+ List<Long> storeFileSizes, HStoreFile file, StoreFileReader r)
+ throws IOException {
+ boolean toBeArchived = false;
+ if (file.isCompactedAway() && !file.isReferencedInReads()) {
+ toBeArchived = true;
+ } else {
+ LOG.info("Can't archive compacted file " + file.getPath()
+ + " because of either isCompactedAway=" + file.isCompactedAway()
+ + " or file has reference, isReferencedInReads=" +
file.isReferencedInReads()
+ + ", refCount=" + r.getRefCount() + ", skipping for now.");
+ if (!file.isCompactedAway()) {
+ LOG.warn("File is sent for archival but is not yet compacted away,
file: {}, refCount: {}",
+ file.getPath(), r.getRefCount());
+ }
+ }
+ if (toBeArchived) {
+ // Even if deleting fails we need not bother as any new scanners won't be
+ // able to use the compacted file as the status is already compactedAway
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Closing and archiving the file {}", file);
+ }
+ // Copy the file size before closing the reader
+ final long length = r.length();
+ r.close(true);
+ // Just close and return
+ filesToRemove.add(file);
+ // Only add the length if we successfully added the file to
`filesToRemove`
+ storeFileSizes.add(length);
+ }
+ return toBeArchived;
+ }
+
+ /**
+ * Notify all observers that set of Readers has changed and Re-Create
scanners
+ *
+ * @throws IOException Something goes wrong with change of scanners
+ */
+ private void notifyChangedReadersObservers() throws IOException {
+ for (ChangedReadersObserver o : this.changedReaderObservers) {
+ // retrieve active store files that are not compacted away
+ Collection<HStoreFile> storeFiles =
this.storeEngine.getStoreFileManager().getStorefiles();
+ // Update readers to close current scanners and re-create new on active
store files
+ o.updateReaders(new ArrayList<>(storeFiles));
Review comment:
What we pass here is not the new files. It is all files under the store
this.storeEngine.getStoreFileManager().getStorefiles()
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
