The following are the notes from today's KIP discussion: - KIP-54 (Sticky Partition Assignment): aims to minimise partition movement so that resource reinitialisation (e.g. caches) is minimised. It is partially sticky and partially fair. Some concerns around the fact that user code for partitionsRevoked and partitionsAssigned would have to be changed to work correctly with this assignment strategy. Good: more complex usage of an assigner that takes advantage of the user data field. Vahid will start the vote.
- KIP-72 (Allow Sizing Incoming Request Queue in Bytes): large requests can kill the broker, no control over how much memory is allocated. Client quotas don't help as damage may already have been done by the time they kick in. There was a discussion on whether it was worth it to avoid the immediate return from select when there was no memory available in the pool. Radai will update the KIP to describe this aspect in more detail as well as the config validation that is performed. - KIP-79 (ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer): we discussed the option of passing multiple timestamps for the same partition in the same request. Becket thinks it's a rare use case and not worth supporting. Gwen said that it would be nice to have, but not essential. We talked about validation of duplicate topics. Becket will check the approach taken by the create topics request and evaluate if it can be adopted here too. PR will be available today and Jason will evaluate if it's feasible to include it in the next release once it's available. The video will be uploaded soon and the link will be posted to: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals Ismael On Mon, Sep 12, 2016 at 4:39 PM, Ismael Juma <ism...@juma.me.uk> wrote: > Hi, Everyone, > > We plan to have a Kafka KIP meeting this coming Tuesday at 11:00am PST. If > you plan to attend but haven't received an invite, please let me know. The > following is the tentative agenda. > > Agenda: > KIP-54: Sticky Partition Assignment Strategy > KIP-72: Allow Sizing Incoming Request Queue in Bytes > > Thanks, > Ismael >
