lucasbru commented on code in PR #23165:
URL: https://github.com/apache/kafka/pull/23165#discussion_r4152867643


##########
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java:
##########
@@ -2268,9 +2266,10 @@ private CoordinatorResult<StreamsGroupHeartbeatResult, 
CoordinatorRecord> stream
             assignmentUpdate = AssignmentUpdate.RECOMPUTE;
         }
 
-        // Check if assignment configurations have changed
-        Map<String, String> currentAssignmentConfigs = 
streamsGroupAssignmentConfigs(groupId);
-        Map<String, String> storedAssignmentConfigs = 
group.lastAssignmentConfigs();
+        // Check if assignment configurations have changed. A group with no 
recorded configs holds the defaults,
+        // so upgrading the broker does not rebalance every group that has 
nothing configured.
+        AssignmentConfigsImpl currentAssignmentConfigs = 
streamsGroupAssignmentConfigs(groupId);
+        AssignmentConfigsImpl storedAssignmentConfigs = 
group.lastAssignmentConfigs();
         if (assignmentUpdate == AssignmentUpdate.NONE && 
!currentAssignmentConfigs.equals(storedAssignmentConfigs)) {

Review Comment:
   DEFAULT is hardcoded to the compiled default (0 / ""), but 
`streamsGroupAssignmentConfigs()` compares against the live broker config 
value, which can differ if `group.streams.num.standby.replicas` (or the 
rack-tags config) is set cluster-wide. Wouldn't that still cause a spurious 
rebalance for a group with no recorded config in that case - just gated on 
"broker default customized" instead of "never recorded"?



##########
group-coordinator/group-coordinator-api/src/main/java/org/apache/kafka/coordinator/group/api/streams/assignor/GroupAssignment.java:
##########
@@ -36,11 +35,11 @@ public class GroupAssignment {
     public GroupAssignment(
         Map<String, MemberAssignment> members
     ) {
-        this.members = Objects.requireNonNull(members);
+        this.members = Map.copyOf(members);

Review Comment:
   Why this change? Map.copyOf() will NPE if any value in the map is null, not 
just the map reference itself - a bit stricter than before for custom 
TaskAssignor implementations. Is that intentional? Also, every caller here 
already builds a fresh, private map just for this constructor, so the copy is 
doing a bit of extra work on top of that.



##########
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java:
##########
@@ -6388,17 +6386,16 @@ public void replay(
             
streamsGroup.setStoredDescriptionTopologyEpoch(value.storedDescriptionTopologyEpoch());
             
streamsGroup.setFailedDescriptionTopologyEpoch(value.failedDescriptionTopologyEpoch());
 
-            if (value.lastAssignmentConfigs() != null) {
-                streamsGroup.setLastAssignmentConfigs(
-                    value.lastAssignmentConfigs().stream()
-                        .collect(Collectors.toMap(
-                            
StreamsGroupMetadataValue.LastAssignmentConfig::key,
-                            
StreamsGroupMetadataValue.LastAssignmentConfig::value
-                        ))
-                );
-            } else {
-                streamsGroup.setLastAssignmentConfigs(Map.of());
-            }
+            // A record without configs (written before they were persisted) 
yields the defaults, so the next
+            // heartbeat only rebalances the group if any effective config 
differs from its default.
+            Map<String, String> lastAssignmentConfigs = 
value.lastAssignmentConfigs() == null
+                ? Map.of()
+                : value.lastAssignmentConfigs().stream()
+                    .collect(Collectors.toMap(
+                        StreamsGroupMetadataValue.LastAssignmentConfig::key,
+                        StreamsGroupMetadataValue.LastAssignmentConfig::value
+                    ));
+            
streamsGroup.setLastAssignmentConfigs(AssignmentConfigsImpl.fromMap(lastAssignmentConfigs));

Review Comment:
   This builds a Map just to immediately hand it to fromMap() - could probably 
construct the AssignmentConfigsImpl directly from the LastAssignmentConfig list 
here instead. Maybe worth the optimization, since replay will be called on 
every GC failover for all groups.



##########
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/streams/assignor/AssignmentConfigsImpl.java:
##########
@@ -52,20 +53,18 @@ public record AssignmentConfigsImpl(
     }
 
     /**
-     * Converts the raw assignment configs computed for the group into the 
typed configs passed to the assignor.
+     * Converts the raw assignment configs recorded for the group into the 
typed configs passed to the assignor.
      */
     public static AssignmentConfigsImpl fromMap(Map<String, String> configs) {
-        // The map is empty when it was replayed from a group metadata record 
written before the last assignment
-        // configs were persisted.
-        if (configs.isEmpty()) {
-            return DEFAULT;
-        }
-        // The rack-aware assignment tags are only recorded when any are 
configured, so an absent value means the
-        // configuration is at its default.
-        String rackAwareAssignmentTags = configs.getOrDefault(
-            RACK_AWARE_ASSIGNMENT_TAGS_CONFIG, 
GroupCoordinatorConfig.STREAMS_GROUP_RACK_AWARE_ASSIGNMENT_TAGS_DEFAULT);
+        // Configs are only recorded when set, so every absent key means the 
configuration is at its default.
+        String numStandbyReplicasConfig = 
configs.get(NUM_STANDBY_REPLICAS_CONFIG);
+        int numStandbyReplicas = numStandbyReplicasConfig != null
+            ? Integer.parseInt(numStandbyReplicasConfig)
+            : 
GroupCoordinatorConfig.STREAMS_GROUP_NUM_STANDBY_REPLICAS_DEFAULT;

Review Comment:
   Could this just reference DEFAULT.numStandbyReplicas() instead of the raw 
constant? Minor, but avoids having two places that need to agree on the default 
value.



##########
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/streams/assignor/AssignmentConfigsImpl.java:
##########
@@ -52,20 +53,18 @@ public record AssignmentConfigsImpl(
     }
 
     /**
-     * Converts the raw assignment configs computed for the group into the 
typed configs passed to the assignor.
+     * Converts the raw assignment configs recorded for the group into the 
typed configs passed to the assignor.
      */
     public static AssignmentConfigsImpl fromMap(Map<String, String> configs) {
-        // The map is empty when it was replayed from a group metadata record 
written before the last assignment
-        // configs were persisted.
-        if (configs.isEmpty()) {
-            return DEFAULT;
-        }
-        // The rack-aware assignment tags are only recorded when any are 
configured, so an absent value means the
-        // configuration is at its default.
-        String rackAwareAssignmentTags = configs.getOrDefault(
-            RACK_AWARE_ASSIGNMENT_TAGS_CONFIG, 
GroupCoordinatorConfig.STREAMS_GROUP_RACK_AWARE_ASSIGNMENT_TAGS_DEFAULT);
+        // Configs are only recorded when set, so every absent key means the 
configuration is at its default.

Review Comment:
   This comment says every absent key means default, but toMap always writes 
num.standby.replicas even when it's at its default.
   
   Which one is it?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to