CalvinConfluent commented on code in PR #15702:
URL: https://github.com/apache/kafka/pull/15702#discussion_r1583446833
##########
metadata/src/main/java/org/apache/kafka/controller/ConfigurationControlManager.java:
##########
@@ -308,6 +328,48 @@ private ApiError validateAlterConfig(ConfigResource
configResource,
return ApiError.NONE;
}
+ void maybeTriggerPartitionUpdateOnMinIsrChange(List<ApiMessageAndVersion>
records) {
+ List<ConfigRecord> minIsrRecords = new ArrayList<>();
+ Map<String, String> topicMap = new HashMap<>();
+ Map<String, String> configRemovedTopicMap = new HashMap<>();
+ records.forEach(record -> {
+ if (MetadataRecordType.fromId(record.message().apiKey()) ==
MetadataRecordType.CONFIG_RECORD) {
+ ConfigRecord configRecord = (ConfigRecord) record.message();
+ if
(configRecord.name().equals(TopicConfig.MIN_IN_SYNC_REPLICAS_CONFIG)) {
+ minIsrRecords.add(configRecord);
+ if (Type.forId(configRecord.resourceType()) == Type.TOPIC)
{
+ if (configRecord.value() == null)
topicMap.put(configRecord.resourceName(), configRecord.value());
+ else
configRemovedTopicMap.put(configRecord.resourceName(), configRecord.value());
+ }
+ }
+ }
+ });
+
+ if (minIsrRecords.isEmpty()) return;
+ if (topicMap.size() == minIsrRecords.size()) {
+ // If all the min isr config updates are on the topic level, we
can trigger a simpler update just on the
+ // updated topics.
+
records.addAll(minIsrConfigUpdatePartitionHandler.addRecordsForMinIsrUpdate(
+ new ArrayList<>(topicMap.keySet()),
+ topicName -> topicMap.get(topicName))
+ );
+ return;
+ }
+
+ // Because it may require multiple layer look up for the min ISR
config value. Build a config data copy and apply
+ // the config updates to it. Use this config copy for the min ISR look
up.
+ Map<ConfigResource, TimelineHashMap<String, String>> configDataCopy =
new HashMap<>(configData);
+ SnapshotRegistry localSnapshotRegistry = new SnapshotRegistry(new
LogContext("dummy-config-update"));
+ for (ConfigRecord record : minIsrRecords) {
+ replayInternal(record, configDataCopy, localSnapshotRegistry);
+ }
Review Comment:
This is the implementation challenge part of this PR. To find the effective
min ISR value, it requires checking topic config -> dynamic broker config ->
default broker config -> ...
Let's say the user updates the default broker config:
1. All the topics could be affected.
2. The effective min ISR values should be recalculated.
3. We need to generate the partition change records along with the config
change records, which means the ReplicationControlManager can't use the regular
methods for the effective min ISR value. The value should be determined by the
config records and the current configs.
I found it easier to make a copy of the configs and apply the min ISR
updates on the copy. Then let the ReplicationControlManager check all the
partitions with the config copy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
