mcvsubbu commented on a change in pull request #6165:
URL: https://github.com/apache/incubator-pinot/pull/6165#discussion_r509405315
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java
##########
@@ -1641,23 +1643,30 @@ public void addNewSegment(String tableName,
SegmentMetadata segmentMetadata, Str
Map<InstancePartitionsType, InstancePartitions> instancePartitionsMap =
Collections
.singletonMap(InstancePartitionsType.OFFLINE, InstancePartitionsUtils
.fetchOrComputeInstancePartitions(_helixZkManager,
offlineTableConfig, InstancePartitionsType.OFFLINE));
- HelixHelper.updateIdealState(_helixZkManager, offlineTableName,
idealState -> {
- assert idealState != null;
- Map<String, Map<String, String>> currentAssignment =
idealState.getRecord().getMapFields();
- if (currentAssignment.containsKey(segmentName)) {
- LOGGER.warn("Segment: {} already exists in the IdealState for table:
{}, do not update", segmentName,
- offlineTableName);
- } else {
- List<String> assignedInstances =
- segmentAssignment.assignSegment(segmentName, currentAssignment,
instancePartitionsMap);
- LOGGER.info("Assigning segment: {} to instances: {} for table: {}",
segmentName, assignedInstances,
- offlineTableName);
- currentAssignment.put(segmentName,
- SegmentAssignmentUtils.getInstanceStateMap(assignedInstances,
SegmentStateModel.ONLINE));
+ if (_tableUpdaterLockMap.get(offlineTableName) == null) {
+ synchronized (_tableUpdaterLockMap) {
+ if (_tableUpdaterLockMap.get(offlineTableName) == null) {
+ _tableUpdaterLockMap.put(offlineTableName, new Object());
Review comment:
Also, we hold the lock while backing off, so a single cross-controller
contention will start building a queue? I think a better solution may to simply
reduce the backoff time and try more often?
Also, we have clusters with 1000s of tables, and in this case, the hash
table will contain all of them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
