mqliang commented on a change in pull request #7585:
URL: https://github.com/apache/pinot/pull/7585#discussion_r732151128
##########
File path:
pinot-common/src/main/java/org/apache/pinot/common/utils/fetcher/HttpSegmentFetcher.java
##########
@@ -35,19 +43,36 @@ protected void doInit(PinotConfiguration config) {
}
@Override
- public void fetchSegmentToLocal(URI uri, File dest)
+ public void fetchSegmentToLocal(URI downloadURI, File dest)
throws Exception {
+ // Create a RoundRobinURIProvider to round robin IP addresses when retry
uploading. Otherwise may always try to
+ // download from a same broken host as: 1) DNS may not RR the IP addresses
2) OS cache the DNS resolution result.
+ RoundRobinURIProvider uriProvider = new RoundRobinURIProvider(downloadURI);
+ _retryCount = Math.max(_retryCount, uriProvider.numAddresses());
+ _logger.info("set retryCount as: {}", _retryCount);
Review comment:
done
##########
File path:
pinot-common/src/main/java/org/apache/pinot/common/utils/fetcher/HttpSegmentFetcher.java
##########
@@ -35,19 +43,36 @@ protected void doInit(PinotConfiguration config) {
}
@Override
- public void fetchSegmentToLocal(URI uri, File dest)
+ public void fetchSegmentToLocal(URI downloadURI, File dest)
throws Exception {
+ // Create a RoundRobinURIProvider to round robin IP addresses when retry
uploading. Otherwise may always try to
+ // download from a same broken host as: 1) DNS may not RR the IP addresses
2) OS cache the DNS resolution result.
+ RoundRobinURIProvider uriProvider = new RoundRobinURIProvider(downloadURI);
+ _retryCount = Math.max(_retryCount, uriProvider.numAddresses());
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
