Dana, That is just what I was looking for, awesome.
That makes more sense now. As for the hunting and pecking for, I have been searching the code to try to make some sense of how the client is intended to work. Again, thanks a lot. Heath -----Original Message----- From: Dana Powers [mailto:dana.pow...@gmail.com] Sent: Tuesday, March 29, 2016 9:06 AM To: users@kafka.apache.org Subject: Re: Documentation I also found the documentation difficult to parse when it came time to implement group APIs. I ended up just reading the client source code and trying api calls until it made sense. My general description from off the top of my head: (1) have all consumers submit a shared protocol_name string* and bytes-serialized protocol_metadata, which includes the topic(s) for subscription. (2) All consumers should be prepared to parse a JoinResponse and identify whether they have been selected as the group leader. (3) The non-leaders go straight to SyncGroupRequest, with group_assignment empty, and will wait get their partition assignments via the SyncGroupResponse. (4) Unlike the others, the consumer-leader will get the agreed protocol_name and all member's metadata in its JoinGroupResponse. (5) The consumer-leader must apply the protocol_name assignment strategy to the member list + metadata (plus the cluster metadata), generating a set of member assignments (member_id -> topic partitions). (6)The consumer-leader then submits that member_assignment data in its SyncGroupRequest. (7) The broker will take the group member_assignment data and split it out into each members SyncGroupResponse (including the consumer-leader). At this point all members have their assignments and can start chugging along. I've tried to encapsulate the group request / response protocol info cleanly into kafka-python, if that helps you: https://github.com/dpkp/kafka-python/blob/1.0.2/kafka/protocol/group.py I'd be willing to help improve the docs if someone points me at the "source of truth". Or feel free to ping on irc #apache-kafka -Dana *for java client these are 'range' or 'roundrobin', but are opaque and can be anything if you aren't interested in joining groups that have heterogeneous clients. related, the protocol_metadata and member_metadata are also opaque and technically can be anything you like. But sticking with the documented metadata specs should in theory allow heterogenous clients to cooperate. On Tue, Mar 29, 2016 at 8:21 AM, Heath Ivie <hi...@autoanything.com> wrote: > Does anyone have better documentation around the group membership APIs? > > The information about the APIs are great at the beginning but get > progressively sparse towards then end. > > I am not finding enough information about the values of the request > fields to join / sync the group. > > Can anyone help me or send me the some additional documentation? > > Heath Ivie > Solutions Architect > > > Warning: This e-mail may contain information proprietary to > AutoAnything Inc. and is intended only for the use of the intended > recipient(s). If the reader of this message is not the intended > recipient(s), you have received this message in error and any review, > dissemination, distribution or copying of this message is strictly > prohibited. If you have received this message in error, please notify > the sender immediately and delete all copies. >
