Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

[Dong Lin]

I am really really sorry for missing the KIP meeting. I am on vacation in
Shanghai. I drank too much and over slept yesterday night.

Thanks much for the KIP meeting notes! I will read through that carefully.

On Thu, Apr 5, 2018 at 10:31 PM, Jeff Chao <jc...@salesforce.com> wrote:

> Hi Jun, same for me. jeffc...@me.com
>
> Thanks,
> Jeff Chao
>
> On Thu, Apr 5, 2018 at 9:06 PM, Jun Rao <j...@confluent.io> wrote:
>
> > Hi, James, Jeff,
> >
> > Added you to the invitation.
> >
> > Jun
> >
> > On Thu, Apr 5, 2018 at 6:11 PM, Jeff Widman <j...@jeffwidman.com> wrote:
> >
> > > Please add me as well: j...@jeffwidman.com
> > >
> > > thanks
> > >
> > > On Thu, Apr 5, 2018 at 5:16 PM, James Cheng <wushuja...@gmail.com>
> > wrote:
> > >
> > > > Jun,
> > > >
> > > > Can you add me as well? wushuja...@gmail.com  > > wushuja...@gmail.com>
> > > >
> > > > Thanks,
> > > > -James
> > > >
> > > > > On Apr 5, 2018, at 1:34 PM, Jun Rao <j...@confluent.io> wrote:
> > > > >
> > > > > Hi, Ted, Vahid,
> > > > >
> > > > > Added you to the invite.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > > On Thu, Apr 5, 2018 at 10:42 AM, Vahid S Hashemian <
> > > > > vahidhashem...@us.ibm.com> wrote:
> > > > >
> > > > >> Hi Jun,
> > > > >>
> > > > >> I used to receive these invites, but didn't get this one.
> > > > >> Please send me an invite. Thanks.
> > > > >>
> > > > >> Regards,
> > > > >> --Vahid
> > > > >>
> > > > >>
> > > > >>
> > > > >> From:   Jun Rao <j...@confluent.io>
> > > > >> To: dev <dev@kafka.apache.org>
> > > > >> Date:   04/05/2018 10:25 AM
> > > > >> Subject:Kafka KIP meeting on Apr. 9 at 9:00am PDT
> > > > >>
> > > > >>
> > > > >>
> > > > >> Hi, Everyone,
> > > > >>
> > > > >> We plan to have a Kafka KIP meeting this coming Monday (Apr. 9)
> at
> > > > 9:00am
> > > > >> PDT. If you plan to attend but haven't received an invite, please
> > let
> > > me
> > > > >> know. The following is the agenda.
> > > > >>
> > > > >> Agenda:
> > > > >> KIP-253: Partition expansion
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> Jun
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> > >
> > > --
> > >
> > > *Jeff Widman*
> > > jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265)
> > > <><
> > >
> >
>

Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

[Jun Rao]

The following are my notes from today's KIP meeting. The link to the video
has been added to
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
.


   - KIP-253 - partition expansion: We discussed a few things. (1) Is it
   useful to backfill a compacted topic? The main use case is to rebuild the
   application states. If the new partition has the existing data, rebuilding
   the state can be done easily by reading from a single partition. Otherwise,
   an application has to read both the child and the parent partition to
   rebuild the state. This is possible, but can be complicated. Jan will do an
   exercise to see how complicated this is. (2) What's the best way to add the
   backfilling support if we want to do it? We can do this on the server side
   or on the client side. The former potentially makes the coordination
   easier. The latter potentially reduces the memory footprint on the
   server for reshuffling. We need to think through how to support EOS message
   format and how to throttle the process to avoid overwhelming the cluster.
   (3) Linear hashing vs doubling partitions. It seems that Linear hashing is
   more general. (4) Partition splitting in Kinesis. This is done differently
   since it doesn't allow customized partitioning. It doesn't support
   compacted topic either. (5) Sticky partition assignment. It could be useful
   to support a partition assignment strategy where the child partition is
   assigned together with the parent partition to a consumer instance so that
   the local state doesn't have to be moved immediately. (6) Consumer callback
   on partition splitting. This could still be useful if the states are
   maintained globally.


Thanks,

Jun


On Thu, Apr 5, 2018 at 10:31 PM, Jeff Chao <jc...@salesforce.com> wrote:

> Hi Jun, same for me. jeffc...@me.com
>
> Thanks,
> Jeff Chao
>
> On Thu, Apr 5, 2018 at 9:06 PM, Jun Rao <j...@confluent.io> wrote:
>
> > Hi, James, Jeff,
> >
> > Added you to the invitation.
> >
> > Jun
> >
> > On Thu, Apr 5, 2018 at 6:11 PM, Jeff Widman <j...@jeffwidman.com> wrote:
> >
> > > Please add me as well: j...@jeffwidman.com
> > >
> > > thanks
> > >
> > > On Thu, Apr 5, 2018 at 5:16 PM, James Cheng <wushuja...@gmail.com>
> > wrote:
> > >
> > > > Jun,
> > > >
> > > > Can you add me as well? wushuja...@gmail.com  > > wushuja...@gmail.com>
> > > >
> > > > Thanks,
> > > > -James
> > > >
> > > > > On Apr 5, 2018, at 1:34 PM, Jun Rao <j...@confluent.io> wrote:
> > > > >
> > > > > Hi, Ted, Vahid,
> > > > >
> > > > > Added you to the invite.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > > On Thu, Apr 5, 2018 at 10:42 AM, Vahid S Hashemian <
> > > > > vahidhashem...@us.ibm.com> wrote:
> > > > >
> > > > >> Hi Jun,
> > > > >>
> > > > >> I used to receive these invites, but didn't get this one.
> > > > >> Please send me an invite. Thanks.
> > > > >>
> > > > >> Regards,
> > > > >> --Vahid
> > > > >>
> > > > >>
> > > > >>
> > > > >> From:   Jun Rao <j...@confluent.io>
> > > > >> To: dev <dev@kafka.apache.org>
> > > > >> Date:   04/05/2018 10:25 AM
> > > > >> Subject:Kafka KIP meeting on Apr. 9 at 9:00am PDT
> > > > >>
> > > > >>
> > > > >>
> > > > >> Hi, Everyone,
> > > > >>
> > > > >> We plan to have a Kafka KIP meeting this coming Monday (Apr. 9)
> at
> > > > 9:00am
> > > > >> PDT. If you plan to attend but haven't received an invite, please
> > let
> > > me
> > > > >> know. The following is the agenda.
> > > > >>
> > > > >> Agenda:
> > > > >> KIP-253: Partition expansion
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> Jun
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> > >
> > > --
> > >
> > > *Jeff Widman*
> > > jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265)
> > > <><
> > >
> >
>

Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

[Jeff Chao]

Hi Jun, same for me. jeffc...@me.com

Thanks,
Jeff Chao

On Thu, Apr 5, 2018 at 9:06 PM, Jun Rao <j...@confluent.io> wrote:

> Hi, James, Jeff,
>
> Added you to the invitation.
>
> Jun
>
> On Thu, Apr 5, 2018 at 6:11 PM, Jeff Widman <j...@jeffwidman.com> wrote:
>
> > Please add me as well: j...@jeffwidman.com
> >
> > thanks
> >
> > On Thu, Apr 5, 2018 at 5:16 PM, James Cheng <wushuja...@gmail.com>
> wrote:
> >
> > > Jun,
> > >
> > > Can you add me as well? wushuja...@gmail.com  > wushuja...@gmail.com>
> > >
> > > Thanks,
> > > -James
> > >
> > > > On Apr 5, 2018, at 1:34 PM, Jun Rao <j...@confluent.io> wrote:
> > > >
> > > > Hi, Ted, Vahid,
> > > >
> > > > Added you to the invite.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > > On Thu, Apr 5, 2018 at 10:42 AM, Vahid S Hashemian <
> > > > vahidhashem...@us.ibm.com> wrote:
> > > >
> > > >> Hi Jun,
> > > >>
> > > >> I used to receive these invites, but didn't get this one.
> > > >> Please send me an invite. Thanks.
> > > >>
> > > >> Regards,
> > > >> --Vahid
> > > >>
> > > >>
> > > >>
> > > >> From:   Jun Rao <j...@confluent.io>
> > > >> To: dev <dev@kafka.apache.org>
> > > >> Date:   04/05/2018 10:25 AM
> > > >> Subject:Kafka KIP meeting on Apr. 9 at 9:00am PDT
> > > >>
> > > >>
> > > >>
> > > >> Hi, Everyone,
> > > >>
> > > >> We plan to have a Kafka KIP meeting this coming Monday (Apr. 9)  at
> > > 9:00am
> > > >> PDT. If you plan to attend but haven't received an invite, please
> let
> > me
> > > >> know. The following is the agenda.
> > > >>
> > > >> Agenda:
> > > >> KIP-253: Partition expansion
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Jun
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > >
> > >
> >
> >
> > --
> >
> > *Jeff Widman*
> > jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265)
> > <><
> >
>

Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

[Jun Rao]

Hi, James, Jeff,

Added you to the invitation.

Jun

On Thu, Apr 5, 2018 at 6:11 PM, Jeff Widman <j...@jeffwidman.com> wrote:

> Please add me as well: j...@jeffwidman.com
>
> thanks
>
> On Thu, Apr 5, 2018 at 5:16 PM, James Cheng <wushuja...@gmail.com> wrote:
>
> > Jun,
> >
> > Can you add me as well? wushuja...@gmail.com  wushuja...@gmail.com>
> >
> > Thanks,
> > -James
> >
> > > On Apr 5, 2018, at 1:34 PM, Jun Rao <j...@confluent.io> wrote:
> > >
> > > Hi, Ted, Vahid,
> > >
> > > Added you to the invite.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Thu, Apr 5, 2018 at 10:42 AM, Vahid S Hashemian <
> > > vahidhashem...@us.ibm.com> wrote:
> > >
> > >> Hi Jun,
> > >>
> > >> I used to receive these invites, but didn't get this one.
> > >> Please send me an invite. Thanks.
> > >>
> > >> Regards,
> > >> --Vahid
> > >>
> > >>
> > >>
> > >> From:   Jun Rao <j...@confluent.io>
> > >> To: dev <dev@kafka.apache.org>
> > >> Date:   04/05/2018 10:25 AM
> > >> Subject:Kafka KIP meeting on Apr. 9 at 9:00am PDT
> > >>
> > >>
> > >>
> > >> Hi, Everyone,
> > >>
> > >> We plan to have a Kafka KIP meeting this coming Monday (Apr. 9)  at
> > 9:00am
> > >> PDT. If you plan to attend but haven't received an invite, please let
> me
> > >> know. The following is the agenda.
> > >>
> > >> Agenda:
> > >> KIP-253: Partition expansion
> > >>
> > >> Thanks,
> > >>
> > >> Jun
> > >>
> > >>
> > >>
> > >>
> > >>
> >
> >
>
>
> --
>
> *Jeff Widman*
> jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265)
> <><
>

Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

[Jeff Widman]

Please add me as well: j...@jeffwidman.com

thanks

On Thu, Apr 5, 2018 at 5:16 PM, James Cheng <wushuja...@gmail.com> wrote:

> Jun,
>
> Can you add me as well? wushuja...@gmail.com <mailto:wushuja...@gmail.com>
>
> Thanks,
> -James
>
> > On Apr 5, 2018, at 1:34 PM, Jun Rao <j...@confluent.io> wrote:
> >
> > Hi, Ted, Vahid,
> >
> > Added you to the invite.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Apr 5, 2018 at 10:42 AM, Vahid S Hashemian <
> > vahidhashem...@us.ibm.com> wrote:
> >
> >> Hi Jun,
> >>
> >> I used to receive these invites, but didn't get this one.
> >> Please send me an invite. Thanks.
> >>
> >> Regards,
> >> --Vahid
> >>
> >>
> >>
> >> From:   Jun Rao <j...@confluent.io>
> >> To: dev <dev@kafka.apache.org>
> >> Date:   04/05/2018 10:25 AM
> >> Subject:Kafka KIP meeting on Apr. 9 at 9:00am PDT
> >>
> >>
> >>
> >> Hi, Everyone,
> >>
> >> We plan to have a Kafka KIP meeting this coming Monday (Apr. 9)  at
> 9:00am
> >> PDT. If you plan to attend but haven't received an invite, please let me
> >> know. The following is the agenda.
> >>
> >> Agenda:
> >> KIP-253: Partition expansion
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >>
> >>
> >>
> >>
>
>


-- 

*Jeff Widman*
jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265)
<><

Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

[James Cheng]

Jun, 

Can you add me as well? wushuja...@gmail.com <mailto:wushuja...@gmail.com>

Thanks,
-James

> On Apr 5, 2018, at 1:34 PM, Jun Rao <j...@confluent.io> wrote:
> 
> Hi, Ted, Vahid,
> 
> Added you to the invite.
> 
> Thanks,
> 
> Jun
> 
> 
> On Thu, Apr 5, 2018 at 10:42 AM, Vahid S Hashemian <
> vahidhashem...@us.ibm.com> wrote:
> 
>> Hi Jun,
>> 
>> I used to receive these invites, but didn't get this one.
>> Please send me an invite. Thanks.
>> 
>> Regards,
>> --Vahid
>> 
>> 
>> 
>> From:   Jun Rao <j...@confluent.io>
>> To: dev <dev@kafka.apache.org>
>> Date:   04/05/2018 10:25 AM
>> Subject:Kafka KIP meeting on Apr. 9 at 9:00am PDT
>> 
>> 
>> 
>> Hi, Everyone,
>> 
>> We plan to have a Kafka KIP meeting this coming Monday (Apr. 9)  at 9:00am
>> PDT. If you plan to attend but haven't received an invite, please let me
>> know. The following is the agenda.
>> 
>> Agenda:
>> KIP-253: Partition expansion
>> 
>> Thanks,
>> 
>> Jun
>> 
>> 
>> 
>> 
>> 

Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

[Jun Rao]

Hi, Ted, Vahid,

Added you to the invite.

Thanks,

Jun


On Thu, Apr 5, 2018 at 10:42 AM, Vahid S Hashemian <
vahidhashem...@us.ibm.com> wrote:

> Hi Jun,
>
> I used to receive these invites, but didn't get this one.
> Please send me an invite. Thanks.
>
> Regards,
> --Vahid
>
>
>
> From:   Jun Rao <j...@confluent.io>
> To: dev <dev@kafka.apache.org>
> Date:   04/05/2018 10:25 AM
> Subject:Kafka KIP meeting on Apr. 9 at 9:00am PDT
>
>
>
> Hi, Everyone,
>
> We plan to have a Kafka KIP meeting this coming Monday (Apr. 9)  at 9:00am
> PDT. If you plan to attend but haven't received an invite, please let me
> know. The following is the agenda.
>
> Agenda:
> KIP-253: Partition expansion
>
> Thanks,
>
> Jun
>
>
>
>
>

Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

[Vahid S Hashemian]

Hi Jun,

I used to receive these invites, but didn't get this one.
Please send me an invite. Thanks.

Regards,
--Vahid



From:   Jun Rao <j...@confluent.io>
To: dev <dev@kafka.apache.org>
Date:   04/05/2018 10:25 AM
Subject:        Kafka KIP meeting on Apr. 9 at 9:00am PDT



Hi, Everyone,

We plan to have a Kafka KIP meeting this coming Monday (Apr. 9)  at 9:00am
PDT. If you plan to attend but haven't received an invite, please let me
know. The following is the agenda.

Agenda:
KIP-253: Partition expansion

Thanks,

Jun

Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

[Ted Yu]

Jun:
I would be interested.

Is there going to be an online meeting ?

Thanks

On Thu, Apr 5, 2018 at 10:24 AM, Jun Rao  wrote:

> Hi, Everyone,
>
> We plan to have a Kafka KIP meeting this coming Monday (Apr. 9)  at 9:00am
> PDT. If you plan to attend but haven't received an invite, please let me
> know. The following is the agenda.
>
> Agenda:
> KIP-253: Partition expansion
>
> Thanks,
>
> Jun
>

Kafka KIP meeting on Apr. 9 at 9:00am PDT

[Jun Rao]

Hi, Everyone,

We plan to have a Kafka KIP meeting this coming Monday (Apr. 9)  at 9:00am
PDT. If you plan to attend but haven't received an invite, please let me
know. The following is the agenda.

Agenda:
KIP-253: Partition expansion

Thanks,

Jun