dajac commented on code in PR #24113:
URL: https://github.com/apache/kafka/pull/24113#discussion_r4513043602


##########
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java:
##########
@@ -3915,11 +3915,12 @@ private CoordinatorResult<Void, CoordinatorRecord> 
handleRegularExpressionsResul
                 if (group.numSubscribedMembers(regex) == 0) continue;
 
                 ResolvedRegularExpression newResolvedRegularExpression = 
entry.getValue();
-                ResolvedRegularExpression oldResolvedRegularExpression = group
-                    .resolvedRegularExpression(regex)
-                    .orElse(ResolvedRegularExpression.EMPTY);
+                Optional<ResolvedRegularExpression> previousResolution = 
group.resolvedRegularExpression(regex);
+                ResolvedRegularExpression oldResolvedRegularExpression = 
previousResolution.orElse(ResolvedRegularExpression.EMPTY);
 
-                if 
(!oldResolvedRegularExpression.topics().equals(newResolvedRegularExpression.topics()))
 {
+                // The first resolution must trigger assignment computation 
even when no topics match.

Review Comment:
   Could we keep the code as it is on trunk and handle the first resolution in 
its own branch? The change would then be limited to the case that it fixes. 
Something like:
   
   ```java
                   ResolvedRegularExpression newResolvedRegularExpression = 
entry.getValue();
                   ResolvedRegularExpression oldResolvedRegularExpression = 
group
                       .resolvedRegularExpression(regex)
                       .orElse(ResolvedRegularExpression.EMPTY);
   
                   if 
(!oldResolvedRegularExpression.topics().equals(newResolvedRegularExpression.topics()))
 {
                       ...
                   } else if (group.resolvedRegularExpression(regex).isEmpty()) 
{
                       // The heartbeat does not bump the group epoch when a 
member subscribes to a
                       // regex that is not resolved yet, so the first 
resolution bumps it even when
                       // the regex matches no topic.
                       bumpGroupEpoch = true;
                   }
   ```



##########
group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java:
##########
@@ -27104,8 +27104,10 @@ memberId2, new MemberAssignmentImpl(mkAssignment(
         );
     }
 
-    @Test
-    public void testConsumerGroupMemberJoinsWithNewRegex() {
+    @ParameterizedTest
+    @ValueSource(booleans = {true, false})
+    public void testConsumerGroupMemberJoinsWithNewRegex(boolean matchesTopic) 
{

Review Comment:
   This covers a new member joining with a regex that matches no topic, which 
has no partitions to give back. Could we also add a test for the case from the 
Jira? Two members subscribe to `foo*`, one switches to a regex that matches no 
topic, the resolution bumps the group epoch and the other member gets the 
partition at its next heartbeat. Today only the integration test covers it.



##########
group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java:
##########
@@ -27207,6 +27209,55 @@ fooTopicName, computeTopicHash(fooTopicName, 
metadataImage)
         );
     }
 
+    @Test
+    public void testConsumerGroupRefreshOfNonMatchingRegexDoesNotBumpEpoch() {
+        String groupId = "regex-refresh-group";
+        String memberId = Uuid.randomUuid().toString();
+        String regex = "nothing-matches-this.*";
+        long groupMetadataHash = computeGroupHash(Map.of());
+
+        GroupMetadataManagerTestContext context = new 
GroupMetadataManagerTestContext.Builder()
+            
.withConfig(GroupCoordinatorConfig.CONSUMER_GROUP_REGEX_REFRESH_INTERVAL_MS_CONFIG,
 10000)
+            .withMetadataImage(new MetadataImageBuilder()
+                .addTopic(Uuid.randomUuid(), "foo", 2)
+                .buildCoordinatorMetadataImage(12345L))
+            .withConsumerGroup(new ConsumerGroupBuilder(groupId, 10)
+                .withMember(new ConsumerGroupMember.Builder(memberId)
+                    .setState(MemberState.STABLE)
+                    .setMemberEpoch(10)
+                    .setPreviousMemberEpoch(10)
+                    .setClientId(DEFAULT_CLIENT_ID)
+                    .setClientHost(DEFAULT_CLIENT_ADDRESS.toString())
+                    .setRebalanceTimeoutMs(5000)
+                    .setSubscribedTopicRegex(regex)
+                    .build())
+                .withResolvedRegularExpression(regex, new 
ResolvedRegularExpression(Set.of(), 12345L, 0L))
+                .withAssignment(memberId, Map.of())
+                .withAssignmentEpoch(10)
+                .withMetadataHash(groupMetadataHash))
+            .build();
+
+        // Refresh an already resolved empty result after the batching 
interval.
+        context.sleep(11000L);
+        context.consumerGroupHeartbeat(new ConsumerGroupHeartbeatRequestData()
+            .setGroupId(groupId)
+            .setMemberId(memberId)
+            .setMemberEpoch(10));
+
+        assertTrue(context.executor.isScheduled(groupId + "-regex"));
+        List<MockCoordinatorExecutor.ExecutorResult<CoordinatorRecord>> tasks 
= context.processTasks();
+        assertEquals(1, tasks.size());
+        assertRecordsEquals(
+            
List.of(GroupCoordinatorRecordHelpers.newConsumerGroupRegularExpressionRecord(
+                groupId,
+                regex,
+                new ResolvedRegularExpression(Set.of(), 12345L, 
context.time.milliseconds())
+            )),
+            tasks.get(0).result().records()
+        );
+        assertEquals(10, 
context.groupMetadataManager.consumerGroup(groupId).groupEpoch());

Review Comment:
   nit: Could we assert `tasks` in one assertion, as in 
`testConsumerGroupMemberJoinsWithNewRegex`? `processTasks` replays the records, 
so the missing epoch record already shows that the group epoch did not change.
   
   ```suggestion
           List<MockCoordinatorExecutor.ExecutorResult<CoordinatorRecord>> 
tasks = context.processTasks();
           assertEquals(
               List.of(
                   new MockCoordinatorExecutor.ExecutorResult<>(
                       groupId + "-regex",
                       new CoordinatorResult<>(List.of(
                           // The resolution is persisted but the group epoch 
is not bumped.
                           
GroupCoordinatorRecordHelpers.newConsumerGroupRegularExpressionRecord(
                               groupId,
                               regex,
                               new ResolvedRegularExpression(Set.of(), 12345L, 
context.time.milliseconds())
                           )
                       ))
                   )
               ),
               tasks
           );
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to