abhishekagarwal87 commented on a change in pull request #11398:
URL: https://github.com/apache/druid/pull/11398#discussion_r668645685
##########
File path:
server/src/main/java/org/apache/druid/segment/loading/SegmentLoaderLocalCacheManager.java
##########
@@ -237,44 +249,83 @@ public File getSegmentFiles(DataSegment segment) throws
SegmentLoadingException
}
/**
- * location may fail because of IO failure, most likely in two cases:<p>
+ * If we have already reserved a location before, probably via {@link
#reserve(DataSegment)}, then only that location
+ * should be tried. Otherwise, we would fetch locations using {@link
StorageLocationSelectorStrategy} and try all
+ * of them one by one till there is success.
+ * Location may fail because of IO failure, most likely in two cases:<p>
* 1. druid don't have the write access to this location, most likely the
administrator doesn't config it correctly<p>
* 2. disk failure, druid can't read/write to this disk anymore
- *
+ * <p>
* Locations are fetched using {@link StorageLocationSelectorStrategy}.
*/
- private StorageLocation loadSegmentWithRetry(DataSegment segment, String
storageDirStr) throws SegmentLoadingException
+ private File loadSegmentWithRetry(DataSegment segment) throws
SegmentLoadingException
{
- Iterator<StorageLocation> locationsIterator = strategy.getLocations();
+ String segmentDir = getSegmentDir(segment);
+
+ // Try the already reserved location. If location has been reserved
outside, then we do not release the location
+ // here and simply delete any downloaded files. That is, we revert
anything we do in this function and nothing else.
+ for (StorageLocation loc : locations) {
+ if (loc.isReserved(segmentDir)) {
+ File storageDir = loc.segmentDirectoryAsFile(segmentDir);
+ boolean success = loadInLocationWithStartMarkerQuietly(loc, segment,
storageDir, false);
+ if (!success) {
+ throw new SegmentLoadingException("Failed to load segment %s in
reserved location [%s]", segment.getId(), loc.getPath().getAbsolutePath());
Review comment:
it mimics the current behavior in following way
Before `reserve` exists
- iterate through each location one by one
- try to download the segment in location
- if fails, alert, clean up the location and move onto next location
- If we can't download in any location, throw an exception to the caller
After introduction of `reserve`, we are only going to try a single location
that has been reserved already. So, we are going to throw exception to the
caller if we can't download to that location. We also do the cleanup and
alerting inside `loadInLocationWithStartMarkerQuietly`, just like we do if no
location is reserved. We, however, do not release the location but let the
caller take care of that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
