heatclub commented on code in PR #10411:
URL: https://github.com/apache/pinot/pull/10411#discussion_r1136182410
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java:
##########
@@ -526,6 +538,9 @@ private File downloadSegmentFromDeepStore(String
segmentName, SegmentZKMetadata
} else {
try {
File tarFile = downloadAndDecrypt(segmentName, zkMetadata,
tempRootDir);
+ if (_isRetrySegmentDownloadUntarFailure) {
Review Comment:
Apart from retry changes in this PR, `downloadSegmentFromDeepStore()` has
only one other flow `downloadAndStreamUntarWithRateLimit()` in which a retry
logic is already present for that path in
`HttpSegmentfetcher.fetchUntarSegmentToLocalStreamed()` for download failure
only:
https://github.com/apache/pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/common/utils/fetcher/HttpSegmentFetcher.java#L108
Should we handle untar failures also there itself ? As both download and
untar happens in that same function for streamed untar with rate limit
The advantage of: `retry logic over the downloadSegmentFromDeepStore()` is
that it required less code changes.
Your thoughts around this ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
