Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Yes, absolutely. I'll reply to the other thread. -Matthias On 3/21/18 8:45 PM, Richard Yu wrote: > Hi Matthias, > > Just wondering, once this KIP goes through. Could I restart my older KIP > to update SubscriptionInfo? > > Thanks > Richard > > On Wed, Mar 21, 2018 at 11:18 AM, Matthias J.

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Sure. Glad you like this KIP. (KIP-258 will be more difficult... It seems that is was a good decision into two). -Matthias On 3/22/18 11:35 AM, James Cheng wrote: > > >> On Mar 21, 2018, at 11:45 PM, Matthias J. Sax wrote: >> >> Yes, it only affects the metadata.

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

> On Mar 21, 2018, at 11:45 PM, Matthias J. Sax wrote: > > Yes, it only affects the metadata. KIP-268 targets metadata upgrade > without store upgrade. > > We can discuss store upgrade further in KIP-258: I think in general, the > upgrade/downgrade behavior might be an

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

so expected！ > 在 2018年3月22日，下午2:16，James Cheng 写道： > > > > >> On Mar 21, 2018, at 11:18 AM, Matthias J. Sax wrote: >> >> Thanks for following up James. >> >>> Is this the procedure that happens during every rebalance? The reason I ask >>> is

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Yes, it only affects the metadata. KIP-268 targets metadata upgrade without store upgrade. We can discuss store upgrade further in KIP-258: I think in general, the upgrade/downgrade behavior might be an issue for upgrading stores. However, this upgrade/downgrade can only happen when upgrading

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

> On Mar 21, 2018, at 11:18 AM, Matthias J. Sax wrote: > > Thanks for following up James. > >> Is this the procedure that happens during every rebalance? The reason I ask >> is that this step: > As long as the leader (before or after upgrade) receives at least >

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Hi Matthias, Just wondering, once this KIP goes through. Could I restart my older KIP to update SubscriptionInfo? Thanks Richard On Wed, Mar 21, 2018 at 11:18 AM, Matthias J. Sax wrote: > Thanks for following up James. > > > Is this the procedure that happens during

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Thanks for following up James. > Is this the procedure that happens during every rebalance? The reason I ask > is that this step: As long as the leader (before or after upgrade) receives at least one old version X Subscription it always sends version Assignment X back (the encoded supported

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Sorry, I see that the VOTE started already, but I have a late question on this KIP. In the "version probing" protocol: > Detailed upgrade protocol from metadata version X to Y (with X >= 1.2): > On startup/rolling-bounce, an instance does not know what version the leader > understands and

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Guozhang, thanks for your comments. 2: I think my main concern is, that 1.2 would be "special" release that everybody need to use to upgrade. As an alternative, we could say that we add the config in 1.2 and keep it for 2 additional releases (1.3 and 1.4) but remove it in 1.5. This gives users

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Yup :) On Mon, Mar 19, 2018 at 10:01 AM, Ted Yu wrote: > bq. some snippet like ProduceRequest / ProduceRequest > > Did you mean ProduceRequest / Response ? > > Cheers > > On Mon, Mar 19, 2018 at 9:51 AM, Guozhang Wang wrote: > > > Hi Matthias, > > > >

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

bq. some snippet like ProduceRequest / ProduceRequest Did you mean ProduceRequest / Response ? Cheers On Mon, Mar 19, 2018 at 9:51 AM, Guozhang Wang wrote: > Hi Matthias, > > About 2: yeah I guess this is a subjective preference. My main concern > about keeping the config

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Hi Matthias, About 2: yeah I guess this is a subjective preference. My main concern about keeping the config / handling code beyond 1.2 release is that it will become a non-cleanable tech debt forever, as fewer and fewer users would need to upgrade from 0.10.x and 1.1.x, and eventually we will

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Thanks for your comments. 1. Because the old leader cannot decode the new Subscription it can only send an empty assignment back. The idea to send empty assignments to all members is interesting. I will try this out in an PR to see how it behaves. 2. I don't see an issue with keeping config

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Hello Matthias, thanks for the KIP. Here are some comments: 1. "For all other instances the leader sends a regular Assignment in version X back." Does that mean the leader will exclude any member of the group whose protocol version that it does not understand? For example, if we have A, B, C with

[DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Hi, I want to propose KIP-268 to allow rebalance metadata version upgrades in Kafka Streams: https://cwiki.apache.org/confluence/display/KAFKA/KIP-268%3A+Simplify+Kafka+Streams+Rebalance+Metadata+Upgrade Looking forward to your feedback. -Matthias signature.asc Description: OpenPGP digital