lucasbru commented on PR #23855: URL: https://github.com/apache/kafka/pull/23855#issuecomment-6288066656
Thanks for the fix! I think we should also fix this on the client. `StreamsGroupHeartbeatRequestManager#getCopartitionGroupsFromStreams` builds the index lists and the list of groups by iterating a `Set`, which is where the nondeterministic order comes from. Could you sort the indexes within each group and then the groups there too, the same way we already sort source/sink/repartition topics in `fromStreamsToHeartbeatRequest`? Clients would then always send a canonical topology. We'd still keep the broker-side normalization permanently, both on join and on replay, because of older clients and already-persisted unsorted records. A test in `StreamsGroupHeartbeatRequestManagerTest` that checks the sorted output would be good too. It might also be worth sorting in `convertToStreamsGroupTopologyRecord`, so that newly written topology records are canonical too. The record format doesn't change, and it means a broker downgrade won't reject newer, sorted clients against a stored record. `fromRecord` would still normalize for groups created before this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
