mjsax commented on code in PR #22999:
URL: https://github.com/apache/kafka/pull/22999#discussion_r3731393838
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/StreamsMembershipManager.java:
##########
@@ -775,6 +776,12 @@ public void
onHeartbeatSuccess(StreamsGroupHeartbeatResponse response) {
return;
}
+ final Set<String> unknownSubtopologies =
unknownSubtopologies(activeTasks, standbyTasks, warmupTasks);
Review Comment:
> this is a guard against a "bad" assignment with a partition not in the
subscription, makes sense.
No, it's about a unknown sub-topology. KS `Topology` is split into
sub-graphs called sub-topologies, which are later instantiated as tasks for
parallelization (based on input topic partitions per sub-topology).
KS does not really care about the partitions/parallelization -- if we create
a task for a sub-topology using a non-existing partition, we just won't read
any data for it (simplified) -- the consumer would never give us any data for
it, but the point is, we don't crash here -- as a matter of fact, we can't even
verify here as we don't have access to the number of partitions.
However, if we are told to create tasks for a non-existing sub-topology (ie,
there is no backing processing sub-graph), we crash, and this guard checks for
exactly this case. If the sub-topology does not exist, we cannot create tasks
for it, and would crash with an exception.
> But we could also have a "bad" assignment if it includes the same
partition twice.
No, that's protected broker side. The assignment (also covering custom
assignors) is represented as `Map<String, Set<Integer>>` thus no sub-topology
(`String` key), nor any partition (`Set<Integer>`) can be a duplicate. -- Of
course, if we would want to guard against a corrupted RPC, it's a different
story...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
