fx19880617 commented on a change in pull request #6165:
URL: https://github.com/apache/incubator-pinot/pull/6165#discussion_r508990839
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java
##########
@@ -1641,23 +1643,30 @@ public void addNewSegment(String tableName,
SegmentMetadata segmentMetadata, Str
Map<InstancePartitionsType, InstancePartitions> instancePartitionsMap =
Collections
.singletonMap(InstancePartitionsType.OFFLINE, InstancePartitionsUtils
.fetchOrComputeInstancePartitions(_helixZkManager,
offlineTableConfig, InstancePartitionsType.OFFLINE));
- HelixHelper.updateIdealState(_helixZkManager, offlineTableName,
idealState -> {
- assert idealState != null;
- Map<String, Map<String, String>> currentAssignment =
idealState.getRecord().getMapFields();
- if (currentAssignment.containsKey(segmentName)) {
- LOGGER.warn("Segment: {} already exists in the IdealState for table:
{}, do not update", segmentName,
- offlineTableName);
- } else {
- List<String> assignedInstances =
- segmentAssignment.assignSegment(segmentName, currentAssignment,
instancePartitionsMap);
- LOGGER.info("Assigning segment: {} to instances: {} for table: {}",
segmentName, assignedInstances,
- offlineTableName);
- currentAssignment.put(segmentName,
- SegmentAssignmentUtils.getInstanceStateMap(assignedInstances,
SegmentStateModel.ONLINE));
+ if (_tableUpdaterLockMap.get(offlineTableName) == null) {
+ synchronized (_tableUpdaterLockMap) {
+ if (_tableUpdaterLockMap.get(offlineTableName) == null) {
+ _tableUpdaterLockMap.put(offlineTableName, new Object());
Review comment:
This won't prevent from other controller, just to improve on the single
controller update. In practice, it will help a lot on idealstate update as
well, as we have only 3 parallel writes to update idealstates instead of
segment push job parallelism.
From client side, we can also try to push to one single controller to avoid
the race condition.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
