Thanks for the KIP Gabby.
Two comments about the newly added interfaces/records.
MJS1: I would like to mark all new public facing API as @Evolving for
now. We might not get it right with the first release, and marking as
evolving would indicate that the API is not stable yet, and we can
introduce breaking changes in minor releases, allowing us to fix errors
quickly in future minor releases.
MJS2: The existing (non public) assignor API returns `GroupAssignment`
which includes warmup task. I believe it's an artifact from the
"classic" case, in which the HA assignor was required to compute
warmups. However, with "streams" we can simplify the assignor and it
would only compute active and standby tasks, while only the GC (ie,
reconciler) will use warmup tasks. Thus, we should change the interface
accordingly, dropping warmup tasks from `MemberAssignment`.
About DJ_02: I have no experience with evolving Java `records` but it
seems they are more similar to `classes`. In general, `interfaces`
provide much higher flexibility than `classes`, so I agree with David
that it might be good to _only_ use `interfaces` and no `records`.
About the configs (DJ_01): I guess I am fine either way, to either
follow the current proposal, or mimic KIP-848 more closely with a single
broker config containing a mix of built-in assignor-names and
fully-qualified class names.
In doubt I would agree with Lucas: the KIP-848 approach is a little bit
cleaner, as having one config seems easier than having two. Let's hear
from David about it.
-Matthias
On 6/16/26 1:55 AM, Lucas Brutschy via dev wrote:
Hi all,
DJ_01: Agree, and I'd push it a little further. On the consumer side
the short name isn't configured at all — group.consumer.assignors is a
single list of built-in names and class names, each entry is resolved
to an instance, and they're mapped by name(); the first entry is the
cluster default. So once the streams assignors are loaded, their short
names already come from TaskAssignor.name(), which makes a separate
group.streams.assignors.names redundant — a single
group.streams.assignors list (default [sticky]) reproduces the
consumer behavior. That also runs against the "mixed name/class
configuration" rejected alternative, since the consumer config is
exactly that mixed list.
DJ_04: I agree with Gabby; "validate all dynamic group configs on the
broker" seems out of scope for this KIP.
LB1: When consumer assignors are loaded from group.consumer.assignors,
instances implementing Configurable get the broker configs passed to
them. Will the custom TaskAssignor loading do the same, so a custom
assignor can read broker-level configuration the way a consumer
assignor can?
Thanks,
Lucas
On Fri, Jun 12, 2026 at 9:53 PM Gabriella Fu via dev
<[email protected]> wrote:
Hi David,
Thanks a lot for the detailed review!
*DJ_01:* That's a fair point. I'll update the KIP to use
group.streams.assignors to align with KIP-848 and remove the misleading
motivation section.
*DJ_02:* Thanks for the suggestion. I went through the KIP-848 interfaces
in org.apache.kafka.coordinator.group.api.assignor as well as the existing
implementations (UniformAssignor, RangeAssignor), and the pattern makes
total sense now. I'll update the input types in the KIP accordingly:
-
GroupSpec will be changed to an interface with per-member accessors,
instead of returning a raw Map<String, AssignmentMemberSpec>.
-
AssignmentMemberSpec will be replaced by two distinct interfaces:
MemberSubscription and MemberAssignment. This will separate the member's
static metadata (processId, instanceId, rackId, clientTags) from its
runtime task assignment (active/standby/warmup tasks), mirroring the
clean split in KIP-848.
*DJ_03:* Thank you for the suggestion. I'll rename it to configs() and
clarify in the text that it will only return configurations relevant to the
assignment.
*DJ_04:*Moving all dynamic group config validation to the broker makes
sense, but that would be a bit beyond the scope of this KIP. In this case,
maybe we should just keep the config like this?
Best,
Gabriella
On Fri, Jun 12, 2026 at 10:54 AM David Jacot <[email protected]> wrote:
Hi Gabriella,
Thanks for the KIP. I have a few high level comments:
DJ_01: The motivation to not use `group.streams.assignors` and follow
the pattern introduced by KIP-848 is pretty weak in my opinion,
especially the "avoids leaking implementation class names into
per-group dynamic configuration" part. The class names wont leak into
the group config as the group config requires the name of the
assignor. From a user perspective, it is exactly the same concept so
using a different way to express it is wrong in my opinion. Should we
just use `group.streams.assignors` to be consistent in our configs?
DJ_02: I would suggest reconsidering the interface of the assignor. We
started with a similar interface in KIP-848 and we realized during the
implementation that using POJOS and returning Maps (e.g. for
members()) was really inflexible. Moreover, I am not sure if using
records is good from an evolutionary point of view. Using interfaces
may be better. It also allows us to wrap internal objects to expose
them to the assignor. Have you looked into the KIP-848's interface and
the various assignors?
DJ_03: Regarding the `assignmentConfigs()`, should we call it
`configs()` as it is already clear that it is for the assignment.
Moreover, I wonder if it is going to return all group configs or only
a subset of them. Could you please clarify in the KIP?
DJ_04: The dynamic group configs validation makes sense to me. Note
that this pattern is not used today. I also wonder whether we should
move the validation of all dynamic group configs there. To be
discussed.
Best,
David
On Thu, Jun 11, 2026 at 8:37 PM Gabriella Fu via dev
<[email protected]> wrote:
Hi all,
I’d like to start the discussion for KIP-1357: Add broker side custom
assignors for "streams" groups
KIP:
https://urldefense.com/v3/__https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=430408758__;!!Ayb5sqE7!t7TDJ8XzjCLooFy9FhJLiHKHu5c7RJf8-41tQuqRykmMLMB2yl4ApQsRMSEVKCIIl0FuvbznfZcDww$
JIRA:
https://urldefense.com/v3/__https://issues.apache.org/jira/browse/KAFKA-20683__;!!Ayb5sqE7!t7TDJ8XzjCLooFy9FhJLiHKHu5c7RJf8-41tQuqRykmMLMB2yl4ApQsRMSEVKCIIl0Fuvbzx-7tAbA$
Summary:
The Streams Rebalance Protocol (KIP-1071) moves task assignment from the
client to the broker, but unlike the classic protocol and KIP-848
consumer
groups, it offers no way to plug in a custom assignor. This KIP closes
that
gap by making the existing streams task assignor interfaces public API
and
adding broker and group configurations so operators can load custom
assignor implementations and select them per group by short name, with no
client-side involvement.
Please let me know your feedback
Thanks,
Gabriella Fu
