Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

2018-04-14 Thread Ron Dagostino
I am unsure if substitution should be supported for just JAAS configs or if we should allow it for cluster/broker/consumer configs. What I think would be helpful would be to boil down this proposal to its most essential requirement, and I think the discussion has helped us arrive at what that

Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion

2018-04-14 Thread Dong Lin
Hey Jeff, Thanks for the review. The scheme for expanding processors of the stateful processing job is described in "Support processor expansion" section in KIP-287 (link

Re: [VOTE] KIP-279: Fix log divergence between leader and follower after fast leader fail over

2018-04-14 Thread Ted Yu
+1 On Sat, Apr 14, 2018 at 3:54 PM, Anna Povzner wrote: > Hi All, > > > I would like to start the vote on KIP-279: Fix log divergence between > leader and follower after fast leader fail over. > > > For reference, here's the KIP wiki: >

Contributor List Request

2018-04-14 Thread Abhishek Sharma
Hi, May I please be added to the contributor list, according to: https://kafka.apache.org/contributing.html Please let me know once I'm added, and I'll start picking up a starter bug on the JIRA board. Thanks! Abhishek

[VOTE] KIP-279: Fix log divergence between leader and follower after fast leader fail over

2018-04-14 Thread Anna Povzner
Hi All, I would like to start the vote on KIP-279: Fix log divergence between leader and follower after fast leader fail over. For reference, here's the KIP wiki:

Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion

2018-04-14 Thread Jeff Chao
Hi, I had a question from Monday's meeting. The current mechanism, as Ray's notes points out, is to create a copy consumer in which a switch over happens when it catches up. Meanwhile, it looks like a producer would still be writing using the old partitioning scheme. Wouldn't there be a case where

Re: [DISCUSS] KIP-276: Add StreamsConfig prefix for different consumers

2018-04-14 Thread Boyang Chen
Thank you Guozhang for the explanation. Ewen, I think your concern has been addressed, do you want share any further concern on this KIP? Thank you! From: Guozhang Wang Sent: Friday, April 13, 2018 12:34 AM To: dev@kafka.apache.org Subject:

Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion

2018-04-14 Thread Dong Lin
Thanks for the notes by Jun and Ray. I have read through the notes. It seems that there are a few questions for the alternative solution by Jan and maybe Jan will answer these questions later? I have summarized the solution, which I previously provided in this thread, in KIP-287, to hopefully

[DISCUSS] KIP-287: Support partition and processor expansion for stateful processing jobs

2018-04-14 Thread Dong Lin
Hi all, I have created KIP-287 to provide high level design to support partition and processor expansion for stateful processing. See https://cwiki.apache.org/confluence/display/KAFKA/KIP-287%3A+Support+partition+and+processor+expansion+for+stateful+processing+jobs . Instead of planning to

Re: [DISCUSS] KIP-286: producer.send() should not block on metadata update

2018-04-14 Thread Dong Lin
Hey Becket, Good point! Thanks for the comment. I have updated the KIP to move the compression to user thread in the common case. Basically user thread can be responsible for compressing and moving messages from per-topic queue to per-partition queue once the metadata is available. Only if IO