Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion

2018-03-04 Thread Guozhang Wang
Hello Dong, thanks a lot for proposing the KIP! I'm in favor of partition splitting, or more generally consistent hashing mechanism over partition rehashing for the reasons that Matthias as summarized well. Another point is that for the downstream stateful consumers that need to reshard their

Re: [DISCUSS] KIP-250 Add Support for Quorum-based Producer Acknowledgment

2018-03-04 Thread Guozhang Wang
Hi Litao, When acks=all is used, we will only require all ISR replicas to ack before returning, your understanding on this part is right. What I meant is, that with acks=all, for the above example with {A, B, C}, if the ISR list still contains all three replicas, we will still require ALL 3 of

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

2018-03-04 Thread Guozhang Wang
Hello Richard, Thanks for the proposed KIP. I have a couple of general comments: 1. I'm not sure if piggy-backing the timeout exception on the existing requestTimeoutMs configured in "request.timeout.ms" is a good idea since a) it is a general config that applies for all types of requests, and

Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion

2018-03-04 Thread Matthias J. Sax
About partition splitting and load balancing: I think there are two cases: 1) one partition is a hot-spot: for this case, splitting a partitions is actually more desirable than adding a new partitions as it ensure the the load of the hot partitions is reduced. Adding partitions does not help

[DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

2018-03-04 Thread Richard Yu
Hi all, I would like to discuss a potential change which would be made to KafkaConsumer: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75974886 Thanks, Richard Yu