sashidhar commented on a change in pull request #8038: Making optimal usage of
multiple segment cache locations
URL: https://github.com/apache/incubator-druid/pull/8038#discussion_r317370713
##########
File path:
server/src/main/java/org/apache/druid/segment/loading/SegmentLoaderLocalCacheManager.java
##########
@@ -175,22 +176,33 @@ public File getSegmentFiles(DataSegment segment) throws
SegmentLoadingException
* location may fail because of IO failure, most likely in two cases:<p>
* 1. druid don't have the write access to this location, most likely the
administrator doesn't config it correctly<p>
* 2. disk failure, druid can't read/write to this disk anymore
+ *
+ * Locations are fetched using {@link StorageLocationSelectorStrategy}.
*/
private StorageLocation loadSegmentWithRetry(DataSegment segment, String
storageDirStr) throws SegmentLoadingException
{
- for (StorageLocation loc : locations) {
+ Iterator<StorageLocation> locations = strategy.getLocations(segment,
storageDirStr);
+ int numLocationsToTry = this.locations.size();
+
+ while (locations.hasNext() && numLocationsToTry > 0) {
+
+ StorageLocation loc = locations.next();
+ numLocationsToTry--; // This is to avoid the cyclic iterator returned
from Round Robin strategy to loop
+ // indefinitely.
+
File storageDir = loc.reserve(storageDirStr, segment);
- if (storageDir != null) {
+
+ if (null != storageDir) {
try {
loadInLocationWithStartMarker(segment, storageDir);
return loc;
}
catch (SegmentLoadingException e) {
try {
log.makeAlert(
- e,
- "Failed to load segment in current location [%s], try next
location if any",
- loc.getPath().getAbsolutePath()
+ e,
Review comment:
fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
