RuiLi8080 commented on a change in pull request #3226: [STORM-3583] Handle
exceptions when AsyncLocalizer tries to get local resources
URL: https://github.com/apache/storm/pull/3226#discussion_r391737966
##########
File path:
storm-server/src/main/java/org/apache/storm/localizer/AsyncLocalizer.java
##########
@@ -456,7 +457,21 @@ public void releaseSlotFor(LocalAssignment assignment,
int port) throws IOExcept
topoConfBlob.removeReference(pna);
}
- for (LocalResource lr : getLocalResources(pna)) {
+ List<LocalResource> localResources = null;
+ try {
+ // Precondition1: Base blob stormconf.ser and stormcode.ser have
been localized
+ // Precondition2: Both these two blob files are fully downloaded
and proper permission been set
+ localResources = getLocalResources(pna);
+ } catch (FileNotFoundException fnfException) {
+ LOG.warn("Local base blobs have not been downloaded yet. "
Review comment:
Added pna info. As for the race condition, I think that could be addressed
by separating executor pools. I did not think of a scenario that race condition
will happen because of catching the exception.
All downloading tasks are CompletableFuture object. File no found happens
when the object is not completed. So here we will forget about it, and
dereference any resources we can, and release the slot for new assignments.
Then the background download threads will succeeds at some points. The
cleanup thread will take care of these orphan files even if they are not
released properly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
