siddharthteotia commented on code in PR #8694:
URL: https://github.com/apache/pinot/pull/8694#discussion_r871940339
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java:
##########
@@ -72,6 +73,8 @@ public abstract class BaseTableDataManager implements
TableDataManager {
private static final Logger LOGGER =
LoggerFactory.getLogger(BaseTableDataManager.class);
protected final ConcurrentHashMap<String, SegmentDataManager>
_segmentDataManagerMap = new ConcurrentHashMap<>();
+ // Semaphore to restrict the maximum number of parallel segment downloads
for a table.
+ private Semaphore _segmentDownloadSemaphore;
Review Comment:
I suggest doing something similar to `SegmentRefreshSemaphore` by wrapping
the Semaphore inside it or can we just reuse it here since it is reused for
both refresh and reload ? May have to change the log message to be more
appropriate
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java:
##########
@@ -119,6 +122,11 @@ public void init(TableDataManagerConfig
tableDataManagerConfig, String instanceI
_resourceTmpDir);
}
_errorCache = errorCache;
+ if (maxParallelSegmentDownloads > 0) {
Review Comment:
We should add info log for acquire
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
