Re: [VOTE] KIP-794: Strictly Uniform Sticky Partitioner

2022-05-05 Thread Guozhang Wang
; > > > Thanks for the KIP and the patience during discussion! > > > > +1 binding from me. > > > > > > > > Luke > > > > > > > > On Thu, Mar 24, 2022 at 3:43 AM Ismael Juma > wrote: > > > > >

Re: [VOTE] KIP-794: Strictly Uniform Sticky Partitioner

2022-05-05 Thread Artem Livshits
; > > On Thu, Mar 24, 2022 at 3:43 AM Ismael Juma wrote: > > > > > > > Thanks for the KIP and for taking the time to address all the > feedback. > > > +1 > > > > (binding) > > > > > > > > Ismael >

Re: [VOTE] KIP-794: Strictly Uniform Sticky Partitioner

2022-05-05 Thread Guozhang Wang
> > > > On Mon, Mar 21, 2022 at 4:52 PM Artem Livshits > > > wrote: > > > > > > > Hi all, > > > > > > > > I'd like to start a vote on > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner > > > > . > > > > > > > > -Artem > > > > > > > > > > -- -- Guozhang

Re: [VOTE] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-26 Thread Lucas Bradstreet
> > > Hi all, > > > > > > I'd like to start a vote on > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner > > > . > > > > > > -Artem > > > > > >

Re: [VOTE] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-23 Thread Luke Chen
PM Artem Livshits > wrote: > > > Hi all, > > > > I'd like to start a vote on > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner > > . > > > > -Artem > > >

Re: [VOTE] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-23 Thread Ismael Juma
Thanks for the KIP and for taking the time to address all the feedback. +1 (binding) Ismael On Mon, Mar 21, 2022 at 4:52 PM Artem Livshits wrote: > Hi all, > > I'd like to start a vote on > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sti

Re: [VOTE] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-23 Thread David Jacot
; > > Hi all, > > > > I'd like to start a vote on > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner > > . > > > > -Artem > >

Re: [VOTE] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-22 Thread Jun Rao
Hi, Artem, Thanks for the KIP. +1 from me. Jun On Mon, Mar 21, 2022 at 4:52 PM Artem Livshits wrote: > Hi all, > > I'd like to start a vote on > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner > . > > -Artem >

[VOTE] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-21 Thread Artem Livshits
Hi all, I'd like to start a vote on https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner . -Artem

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-17 Thread Artem Livshits
art voting in the next couple of days. -Artem On Mon, Mar 14, 2022 at 6:19 PM Artem Livshits wrote: > Hi Jun, > > 33. Sounds good. Updated the KIP. > > -Artem > > On Mon, Mar 14, 2022 at 5:45 PM Jun Rao wrote: > >> Hi, Artem, >> >> 33. We introduc

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-14 Thread Artem Livshits
Hi Jun, 33. Sounds good. Updated the KIP. -Artem On Mon, Mar 14, 2022 at 5:45 PM Jun Rao wrote: > Hi, Artem, > > 33. We introduced onNewBatch() primarily for the sticky partitioner. It > seems to be a very subtle API to explain and to use properly. If we can't > find any c

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-14 Thread Jun Rao
Hi, Artem, 33. We introduced onNewBatch() primarily for the sticky partitioner. It seems to be a very subtle API to explain and to use properly. If we can't find any convincing usage, it's probably better to deprecate it so that we could keep the API clean. Thanks, Jun On Mon, Mar 14, 2022

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-14 Thread Artem Livshits
Hi Jun, 33. That's an interesting point. Technically, onNewBatch is just a way to pass some signal to the partitioner, the sticky partitioner uses this signal that is suboptimal, in theory someone could use it for something else -Artem On Mon, Mar 14, 2022 at 9:11 AM Jun Rao wrote: >

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-14 Thread Jun Rao
gt; > questions > > > > > > > > > > are > > > > > > > > > > > > > > slightly > > > > > > > > > > > > >

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-10 Thread Artem Livshits
; > > > > > on > > > > > > > > > > > > the > > > > > > > > > > > > > > broker's state at each moment in time. But > because > > > > it's > > > > > > > smooth, > > > > > > > > > it > > > > > > > > > > > may > > > > > > > > > > > > > not > > > > > > > > > > > > > > be reactiv

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-10 Thread Jun Rao
ion.availability.timeout.ms provides an > > > > opportunity > > > > > to > > > > > > > > tune > > > > > > > > > > > > > adaptiveness. > > > > > > > > > > > > > > > > > > > > > > > > > > (Jun) Can we just not assign partitio

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-10 Thread Artem Livshits
> > feel > > > > > > > that > > > > > > > > > the > > > > > > > > > > > > algorithm should try to be fair in general and use > > > smoother > > > > > > > signals > > > > > > > > > by > > > > > >

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-10 Thread Jun Rao
; > info > > > > > > rather > > > > > > > > > than a > > > > > > > > > > > threshold (so if all brokers are heavily, but equally > > > loaded, > > > > > > they > > > > > > > > will > > > &

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-09 Thread Artem Livshits
gt; > > > > good > > > > > > > > > > to try to be fair under normal circumstances, so if > > normally > > > > > > brokers > > > > > > > > can > > > > > > > > > > respond un

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-04 Thread Luke Chen
on the environment and app > requirements, > > > > hence > > > > > > > it's > > > > > > > > > configurable. > > > > > > > > > > > > > > > > > > 10. Added a statement at the beginning of the proposed > > changes. > > > > > > > > > > > > > > > > > > -Artem > > > > > > >

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-04 Thread Ismael Juma
with Luke that having the partitioner returning > -1 > > > is > > > > > kind > > > > > > > of > > > > > > > > > weird. Could we just change the implementation of > > > > > DefaultPa

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-04 Thread David Jacot
ar why we need > > > > > > > > partition.availability.timeout.ms. The KIP says the broker > > > "would > > > > > not > > > > > > be > > > > > > > > chosen until the broker is able to accept the next ready batch &

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-03 Thread Artem Livshits
> > > > > > StickyPartitioner when the key is specified. Since this KIP > > > improves > > > > > upon > > > > > > > StickyPartitioner, it would be useful to make the new behavior > > the > >

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-03 Thread Jun Rao
M Luke Chen > > wrote: > > > > > > > > > > > > > Hi Artem, > > > > > > > > > > > > > > Also, one more thing I think you need to know. > > > > > > > As this bug KAFKA-7572 < > > > &g

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-03-03 Thread Artem Livshits
> > > > > > > > > > > > > > Thanks for the update. I have some questions about it: > > > > > > > > > > > > > > 1. Could you explain why you need the `partitioner` return -1? > In > > > > which

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-02-28 Thread Jun Rao
it's > > > > > > the same as current behavior for backward compatibility, right? > You > > > > > should > > > > > > mention it. > > > > > > 3. I'm thinking we can have a threshold to the > >

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-02-25 Thread Artem Livshits
> > > a > > > > > threshold to 95% (for example), we can know previous 15.5KB already > > > > exceeds > > > > > the threshold so that we can directly create new batch for next > > > records. > > > &g

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-02-23 Thread Jun Rao
nefit of having `partition.availability.timeout.ms` config. In > > > > short, you want to make the partitioner take the broker load into > > > > consideration. We can just improve that in the queuing logic (and you > > > > already did it). Why should we add the

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-02-18 Thread Artem Livshits
8:57 AM Artem Livshits > > > wrote: > > > > > >> Hello, > > >> > > >> Please add your comments about the KIP. If there are no > considerations, > > >> I'll put it up for vote in the next few days. > > >> >

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-02-17 Thread Jun Rao
n Wed, Feb 16, 2022 at 8:57 AM Artem Livshits > > wrote: > > > >> Hello, > >> > >> Please add your comments about the KIP. If there are no considerations, > >> I'll put it up for vote in the next few days. > >> > >> -Artem > >> &

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-02-16 Thread Luke Chen
up for vote in the next few days. >> >> -Artem >> >> On Mon, Feb 7, 2022 at 6:01 PM Artem Livshits >> wrote: >> >> > Hello, >> > >> > After trying a few prototypes, I've made some changes to the public >> > interface. Plea

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-02-15 Thread Luke Chen
> > interface. Please see the updated document > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner > > . > > > > -Artem > > > > On Thu, Nov 4, 2021 at 10:37 AM Artem Livshits > > wrote: > > >

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-02-15 Thread Artem Livshits
Please see the updated document > https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner > . > > -Artem > > On Thu, Nov 4, 2021 at 10:37 AM Artem Livshits > wrote: > >> Hello, >> >> This is the discussion thread fo

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2022-02-07 Thread Artem Livshits
Hello, After trying a few prototypes, I've made some changes to the public interface. Please see the updated document https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner . -Artem On Thu, Nov 4, 2021 at 10:37 AM Artem Livshits wrote: > He

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2021-11-19 Thread Artem Livshits
method that takes a callback that can be used to estimate record size. I've updated the KIP correspondingly https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner -Artem On Mon, Nov 8, 2021 at 5:42 PM Artem Livshits wrote: > Hi Luke, Justine, > >

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2021-11-08 Thread Artem Livshits
n opportunity to hit 5 in-flight and start accumulating sooner. KIP-782 will make this even better, so batches could also grow beyond 16KB if production rate is high. -Artem On Mon, Nov 8, 2021 at 11:56 AM Justine Olshan wrote: > Hi Artem, > Thanks for working on improving the Sticky Partition

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2021-11-08 Thread Justine Olshan
Hi Artem, Thanks for working on improving the Sticky Partitioner! I had a few questions about this portion: *The batching will continue until either an in-flight batch completes or we hit the N bytes and move to the next partition. This way it takes just 5 records to get to batching mode, not 5

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2021-11-08 Thread Luke Chen
Thanks Artem, It's much better now. I've got your idea. In KIP-480: Sticky Partitioner, we'll change partition (call partitioner) when either 1 of below condition match 1. the batch is full 2. when linger.ms is up But, you are changing the definition, into a "partitioner.sticky.batch.size&

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2021-11-05 Thread Artem Livshits
2. In the "Proposed change" section, you take an example to use > "ClassicDefaultPartitioner", is that referring to the current default > sticky partitioner? I think it'd better you name your proposed partition > with a different name for distinguish between the de

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2021-11-04 Thread Luke Chen
plain more in KIP (with an example will be better as suggestion (4)) 2. In the "Proposed change" section, you take an example to use "ClassicDefaultPartitioner", is that referring to the current default sticky partitioner? I think it'd better you name your proposed partition with a

[DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

2021-11-04 Thread Artem Livshits
Hello, This is the discussion thread for https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner . The proposal is a bug fix for https://issues.apache.org/jira/browse/KAFKA-10888, but it does include a client config change, therefore we have a KIP

Re: Sticky Partitioner

2020-12-02 Thread Justine Olshan
Hi Evelyn, Thanks for taking a look at improving the sticky partitioner! These edge cases seem like they would cause quite a bit a trouble. I think the idea to check for max.in.flight.requests.per.connection is a good one, but one concern I have is how this information will be available

Sticky Partitioner

2020-11-30 Thread Eevee
Hi all, I've noticed a couple edge cases in the Sticky Partitioner and I'd like to discuss introducing a new KIP to fix it. Behavior 1. Low throughput producers The first edge case occurs when a broker becomes temporarily unavailable for a period less then replica.lag.time.max.ms. If you

[jira] [Resolved] (KAFKA-8601) Producer Improvement: Sticky Partitioner

2019-08-24 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-8601. --- Resolution: Fixed > Producer Improvement: Sticky Partitio

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-26 Thread Jun Rao
Hi, Justine, Thanks for the updated KIP. The new interface seems cleaner to me. +1 Jun On Fri, Jul 26, 2019 at 4:14 PM Justine Olshan wrote: > Hello all, > I've just added the proposed changes to the KIP page > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-26 Thread Justine Olshan
Hello all, I've just added the proposed changes to the KIP page https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner . The PR has been updated as well. https://github.com/apache/kafka/pull/6997. The idea is that there will just be a separate void method to change

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-26 Thread Justine Olshan
t; > > > > > > Thank you, > > > > > > Justine > > > > > > > > > > > > On Sun, Jul 14, 2019 at 3:07 AM M. Manna > > wrote: > > > > > > > > > >

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-26 Thread Jun Rao
2019 at 3:07 AM M. Manna > wrote: > > > > > > > > > > > +1(na) > > > > > > > > > > > > On Sat, 13 Jul 2019 at 22:17, Stanislav Kozlovski < > > > > > stanis...@confluent.io> > >

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-19 Thread Justine Olshan
; > > > > > On Sat, 13 Jul 2019 at 22:17, Stanislav Kozlovski < > > > > stanis...@confluent.io> > > > > > wrote: > > > > > > > > > > > +1 (non-binding) > > > > > > >

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-19 Thread Bill Bejeck
; > > > > > > > > > Thanks! > > > > > > > > > > On Fri, Jul 12, 2019 at 6:02 PM Gwen Shapira > > > wrote: > > > > > > > > > > > +1 (binding) > > > > > > > > > > > > Thank you for

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-19 Thread Mickael Maison
Thanks! > > > > > > > > On Fri, Jul 12, 2019 at 6:02 PM Gwen Shapira > > wrote: > > > > > > > > > +1 (binding) > > > > > > > > > > Thank you for the KIP. This was long awaited. > > > > > &

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-18 Thread David Arthur
> > +1 (non-binding) > > > > > > Thanks! > > > > > > On Fri, Jul 12, 2019 at 6:02 PM Gwen Shapira > wrote: > > > > > > > +1 (binding) > > > > > > > > Thank you for the KIP. This was long awaited. > >

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-17 Thread Justine Olshan
> > On Tue, Jul 9, 2019 at 5:15 PM Justine Olshan > > > wrote: > > > > > > > > Hello all, > > > > > > > > I'd like to start the vote for KIP-480 : Sticky Partitioner. > > > > > > > > > > https://cwiki.

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-14 Thread M. Manna
Jul 9, 2019 at 5:15 PM Justine Olshan > > wrote: > > > > > > Hello all, > > > > > > I'd like to start the vote for KIP-480 : Sticky Partitioner. > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner >

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-13 Thread Stanislav Kozlovski
+1 (non-binding) Thanks! On Fri, Jul 12, 2019 at 6:02 PM Gwen Shapira wrote: > +1 (binding) > > Thank you for the KIP. This was long awaited. > > On Tue, Jul 9, 2019 at 5:15 PM Justine Olshan > wrote: > > > > Hello all, > > > > I'd like to start

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-07-13 Thread Jun Rao
he first PR, of course.) It would be an > > option > > > for people who wanted to configure this behavior. > > > > > > best, > > > Colin > > > > > > > > > On Wed, Jul 10, 2019, at 08:48, Justine Olshan wrote: > > > > Hi M, > > >

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-07-12 Thread Justine Olshan
add a new, separate StickyRoundRobinPartitioner > class > > > to > > > > KIP-480 which just implemented the sticky behavior regardless of > whether > > > > the key was null. That seems pretty easy to add (and it wouldn't > have to > > > > implemented

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-07-12 Thread Colin McCabe
; > > for people who wanted to configure this behavior. > > > > > > best, > > > Colin > > > > > > > > > On Wed, Jul 10, 2019, at 08:48, Justine Olshan wrote: > > > > Hi M, > > > > > > > > I'm a little con

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-12 Thread Gwen Shapira
+1 (binding) Thank you for the KIP. This was long awaited. On Tue, Jul 9, 2019 at 5:15 PM Justine Olshan wrote: > > Hello all, > > I'd like to start the vote for KIP-480 : Sticky Partitioner. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitione

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-07-12 Thread Justine Olshan
o > > implemented right away in the first PR, of course.) It would be an > option > > for people who wanted to configure this behavior. > > > > best, > > Colin > > > > > > On Wed, Jul 10, 2019, at 08:48, Justine Olshan wrote: > > > Hi M, > > >

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-07-11 Thread Jun Rao
an by extending the behavior on to > the > > RoundRobinPartitioner. > > The sticky partitioner plans to remove the round-robin behavior from > > records with no keys. Instead of sending them to each partition in order, > > it sends them all to the same partition until the b

Re: [VOTE] KIP-480 : Sticky Partitioner

2019-07-11 Thread Colin McCabe
: > Hello all, > > I'd like to start the vote for KIP-480 : Sticky Partitioner. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner > > Thank you, > Justine Olshan >

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-07-11 Thread Colin McCabe
extending the behavior on to the > RoundRobinPartitioner. > The sticky partitioner plans to remove the round-robin behavior from > records with no keys. Instead of sending them to each partition in order, > it sends them all to the same partition until the batch is sent. > I don't think

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-07-10 Thread Justine Olshan
Hi M, I'm a little confused by what you mean by extending the behavior on to the RoundRobinPartitioner. The sticky partitioner plans to remove the round-robin behavior from records with no keys. Instead of sending them to each partition in order, it sends them all to the same partition until

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-07-10 Thread M. Manna
; > > the > > > >>> producer stops sending to it, which puts even more load on the > > > remaining > > > >>> partitions, which are even more likely to fail then, etc. It also > will > > > >>> create unbalanced load patte

[VOTE] KIP-480 : Sticky Partitioner

2019-07-09 Thread Justine Olshan
Hello all, I'd like to start the vote for KIP-480 : Sticky Partitioner. https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner Thank you, Justine Olshan

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-07-09 Thread Colin McCabe
which are even more likely to fail then, etc. It also will > > >>> create unbalanced load patterns on the consumers. > > >>> > > >>> > > > >>> > 2. If there's no measurable performance difference, I agree with > > >>>

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-07-09 Thread M. Manna
load patterns on the consumers. > >>> > >>> > > >>> > 2. If there's no measurable performance difference, I agree with > >>> Stanislav > >>> > that Optional would be better than Integer. >

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-07-08 Thread Justine Olshan
er than Integer. >>> > >>> > 3. We should include the javadoc for the newly introduced method that >>> > specifies it and its parameters. In particular, it would good to >>> specify if >>> > it gets called when an explicit partition id has been provided. >&g

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-07-02 Thread Justine Olshan
particular, it would good to >> specify if >> > it gets called when an explicit partition id has been provided. >> >> Agreed. >> >> best, >> Colin >> >> > >> > Ismael >> > >> > On Mon, Jun 24, 2019, 2:04 PM Justine Olshan >> wrote: >> > >> > > Hello, >> > > This is the discussion thread for KIP-480: Sticky Partitioner. >> > > >> > > >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner >> > > >> > > Thank you, >> > > Justine Olshan >> > > >> > >> >

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-28 Thread Justine Olshan
f > > it gets called when an explicit partition id has been provided. > > Agreed. > > best, > Colin > > > > > Ismael > > > > On Mon, Jun 24, 2019, 2:04 PM Justine Olshan > wrote: > > > > > Hello, > > > This is the discuss

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-27 Thread Colin McCabe
when an explicit partition id has been provided. Agreed. best, Colin > > Ismael > > On Mon, Jun 24, 2019, 2:04 PM Justine Olshan wrote: > > > Hello, > > This is the discussion thread for KIP-480: Sticky Partitioner. > > > > > > https://cwiki.apach

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-27 Thread Justine Olshan
are safer in general. There could be >> some downsides too, so worth thinking about the trade-offs. >> >> Ismael >> >> On Thu, Jun 27, 2019, 10:24 AM Justine Olshan >> wrote: >> >> > Ismael, >> > >> > Thanks for the feedback! >>

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-27 Thread Justine Olshan
> > > Thanks for the feedback! > > > > For 1, currently the sticky partitioner favors "available partitions." > From > > my understanding, these are partitions that are not under-replicated. If > > that is not the same, please let me know. > > A

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-27 Thread Ismael Juma
On Thu, Jun 27, 2019, 10:24 AM Justine Olshan wrote: > Ismael, > > Thanks for the feedback! > > For 1, currently the sticky partitioner favors "available partitions." From > my understanding, these are partitions that are not under-replicated. If > that is n

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-27 Thread Justine Olshan
Ismael, Thanks for the feedback! For 1, currently the sticky partitioner favors "available partitions." From my understanding, these are partitions that are not under-replicated. If that is not the same, please let me know. As for 2, I've switched to Optional, and the few tests I've

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-27 Thread Ismael Juma
ussion thread for KIP-480: Sticky Partitioner. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner > > Thank you, > Justine Olshan >

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-26 Thread Justine Olshan
Colin McCabe < > > cmcc...@apache.org> > > > >> wrote: > > > >> > > > > > > >> > > > > Hi Justine, > > > >> > > > > > > > >> > > > > Thanks for the K

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-26 Thread Stanislav Kozlovski
> > > >> > > > > Thanks for the KIP. This looks great! > > >> > > > > > > >> > > > > In one place in the KIP, you write: "Remove > > >> > > > > testRoundRobinWithUnavailablePartitions

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-25 Thread Colin McCabe
binWithUnavailablePartitions() and testRoundRobin() > >> since > >> > > the > >> > > > > round robin functionality of the partitioner has been removed." > >> You > >> > > can > >> > > > > skip this and similar lines. We don't need to describe changes to > >> > > internal > >> > > > > test classes in the KIP since they're not visible to users or > >> external > >> > > > > developers. > >> > > > > > >> > > > > It seems like maybe the performance tests should get their own > >> section. > >> > > > > Right now, the way the layout is makes it look like they are part > >> of > >> > > the > >> > > > > "Compatibility, Deprecation, and Migration Plan" > >> > > > > > >> > > > > best, > >> > > > > Colin > >> > > > > > >> > > > > > >> > > > > On Mon, Jun 24, 2019, at 14:04, Justine Olshan wrote: > >> > > > > > Hello, > >> > > > > > This is the discussion thread for KIP-480: Sticky Partitioner. > >> > > > > > > >> > > > > > > >> > > > > > >> > > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner > >> > > > > > > >> > > > > > Thank you, > >> > > > > > Justine Olshan > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-25 Thread Justine Olshan
e partitioner has been removed." >> You >> > > can >> > > > > skip this and similar lines. We don't need to describe changes to >> > > internal >> > > > > test classes in the KIP since they're not visible to users or >> external >> > > > > developers. >> > > > > >> > > > > It seems like maybe the performance tests should get their own >> section. >> > > > > Right now, the way the layout is makes it look like they are part >> of >> > > the >> > > > > "Compatibility, Deprecation, and Migration Plan" >> > > > > >> > > > > best, >> > > > > Colin >> > > > > >> > > > > >> > > > > On Mon, Jun 24, 2019, at 14:04, Justine Olshan wrote: >> > > > > > Hello, >> > > > > > This is the discussion thread for KIP-480: Sticky Partitioner. >> > > > > > >> > > > > > >> > > > > >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner >> > > > > > >> > > > > > Thank you, >> > > > > > Justine Olshan >> > > > > > >> > > > > >> > > > >> > > >> > >> >

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-25 Thread Justine Olshan
test classes in the KIP since they're not visible to users or > external > > > > > developers. > > > > > > > > > > It seems like maybe the performance tests should get their own > section. > > > > > Right now, the way the layout is makes it look like they are part > of > > > the > > > > > "Compatibility, Deprecation, and Migration Plan" > > > > > > > > > > best, > > > > > Colin > > > > > > > > > > > > > > > On Mon, Jun 24, 2019, at 14:04, Justine Olshan wrote: > > > > > > Hello, > > > > > > This is the discussion thread for KIP-480: Sticky Partitioner. > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner > > > > > > > > > > > > Thank you, > > > > > > Justine Olshan > > > > > > > > > > > > > > > > > > > > >

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-25 Thread Colin McCabe
d similar lines. We don't need to describe changes to > > internal > > > > test classes in the KIP since they're not visible to users or external > > > > developers. > > > > > > > > It seems like maybe the performance tests should

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-25 Thread Justine Olshan
> the > > > "Compatibility, Deprecation, and Migration Plan" > > > > > > best, > > > Colin > > > > > > > > > On Mon, Jun 24, 2019, at 14:04, Justine Olshan wrote: > > > > Hello, > > > > This is the discussion thread for KIP-480: Sticky Partitioner. > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner > > > > > > > > Thank you, > > > > Justine Olshan > > > > > > > > > >

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-25 Thread Colin McCabe
st classes in the KIP since they're not visible to users or external > > > > developers. > > > > > > > > It seems like maybe the performance tests should get their own section. > > > > Right now, the way the layout is makes it look like they are part of > >

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-25 Thread Justine Olshan
t; It seems like maybe the performance tests should get their own section. > > > Right now, the way the layout is makes it look like they are part of > the > > > "Compatibility, Deprecation, and Migration Plan" > > > > > > best, > > > Colin &g

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-25 Thread Colin McCabe
makes it look like they are part of the > > "Compatibility, Deprecation, and Migration Plan" > > > > best, > > Colin > > > > > > On Mon, Jun 24, 2019, at 14:04, Justine Olshan wrote: > > > Hello, > > > This is the discussion thread for KIP-480: Sticky Partitioner. > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner > > > > > > Thank you, > > > Justine Olshan > > > > > >

[jira] [Created] (KAFKA-8601) Producer Improvement: Sticky Partitioner

2019-06-25 Thread Justine Olshan (JIRA)
Justine Olshan created KAFKA-8601: - Summary: Producer Improvement: Sticky Partitioner Key: KAFKA-8601 URL: https://issues.apache.org/jira/browse/KAFKA-8601 Project: Kafka Issue Type

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-24 Thread Boyang Chen
, Justine Olshan wrote: > > Hello, > > This is the discussion thread for KIP-480: Sticky Partitioner. > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner > > > > Thank you, > > Justine Olshan > > >

Re: [DISCUSS] KIP-480 : Sticky Partitioner

2019-06-24 Thread Colin McCabe
, and Migration Plan" best, Colin On Mon, Jun 24, 2019, at 14:04, Justine Olshan wrote: > Hello, > This is the discussion thread for KIP-480: Sticky Partitioner. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner > > Thank you, > Justine Olshan >

[DISCUSS] KIP-480 : Sticky Partitioner

2019-06-24 Thread Justine Olshan
Hello, This is the discussion thread for KIP-480: Sticky Partitioner. https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner Thank you, Justine Olshan