Hello Justine, I have one item I wanted to discuss.
We are currently in review stage for KAFKA-3333 where we can choose always RoundRobin regardless of null/usable key. If I understood this KIP motivation correctly, you are still honouring how the hashing of key works for DefaultPartitioner. Would you say that having an always "Round-Robin" partitioning with "Sticky" assignment (efficient batching of records for a partition) doesn't deviate from your original intention? Thanks, On Tue, 9 Jul 2019 at 01:00, Justine Olshan <jols...@confluent.io> wrote: > Hello all, > > If there are no more comments or concerns, I would like to start the vote > on this tomorrow afternoon. > > However, if there are still topics to discuss, feel free to bring them up > now. > > Thank you, > Justine > > On Tue, Jul 2, 2019 at 4:25 PM Justine Olshan <jols...@confluent.io> > wrote: > > > Hello again, > > > > Another update to the interface has been made to the KIP. > > Please let me know if you have any feedback! > > > > Thank you, > > Justine > > > > On Fri, Jun 28, 2019 at 2:52 PM Justine Olshan <jols...@confluent.io> > > wrote: > > > >> Hi all, > >> I made some changes to the KIP. > >> The idea is to clean up the code, make behavior more explicit, provide > >> more flexibility, and to keep default behavior the same. > >> > >> Now we will change the partition in onNewBatch, and specify the > >> conditions for this function call (non-keyed values, no explicit > >> partitions) in willCallOnNewBatch. > >> This clears up some of the issues with the implementation. I'm happy to > >> hear further opinions and discuss this change! > >> > >> Thank you, > >> Justine > >> > >> On Thu, Jun 27, 2019 at 2:53 PM Colin McCabe <cmcc...@apache.org> > wrote: > >> > >>> On Thu, Jun 27, 2019, at 01:31, Ismael Juma wrote: > >>> > Thanks for the KIP Justine. It looks pretty good. A few comments: > >>> > > >>> > 1. Should we favor partitions that are not under replicated? This is > >>> > something that Netflix did too. > >>> > >>> This seems like it could lead to cascading failures, right? If a > >>> partition becomes under-replicated because there is too much traffic, > the > >>> producer stops sending to it, which puts even more load on the > remaining > >>> partitions, which are even more likely to fail then, etc. It also will > >>> create unbalanced load patterns on the consumers. > >>> > >>> > > >>> > 2. If there's no measurable performance difference, I agree with > >>> Stanislav > >>> > that Optional would be better than Integer. > >>> > > >>> > 3. We should include the javadoc for the newly introduced method that > >>> > specifies it and its parameters. In particular, it would good to > >>> specify if > >>> > it gets called when an explicit partition id has been provided. > >>> > >>> Agreed. > >>> > >>> best, > >>> Colin > >>> > >>> > > >>> > Ismael > >>> > > >>> > On Mon, Jun 24, 2019, 2:04 PM Justine Olshan <jols...@confluent.io> > >>> wrote: > >>> > > >>> > > Hello, > >>> > > This is the discussion thread for KIP-480: Sticky Partitioner. > >>> > > > >>> > > > >>> > > > >>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner > >>> > > > >>> > > Thank you, > >>> > > Justine Olshan > >>> > > > >>> > > >>> > >> >
