RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2019-01-29 Thread ChienHsing Wu
well enough. --CH -Original Message- From: Colin McCabe Sent: Tuesday, January 29, 2019 4:00 PM To: dev@kafka.apache.org Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer On Mon, Jan 28, 2019, at 06:57, ChienHsing Wu wrote

Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2019-01-29 Thread Colin McCabe
ponses below: > > -Original Message- > From: ChienHsing Wu > Sent: Monday, January 21, 2019 12:47 PM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message > Consumption Across Partitions in KafkaConsumer > > Hi all, not s

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2019-01-28 Thread ChienHsing Wu
in KafkaConsumer Hi all, not sure what to do next as weeks have gone by, guys. --CH -Original Message- From: ChienHsing Wu Sent: Monday, January 14, 2019 9:13 AM To: dev@kafka.apache.org Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2019-01-21 Thread ChienHsing Wu
Hi all, not sure what to do next as weeks have gone by, guys. --CH -Original Message- From: ChienHsing Wu Sent: Monday, January 14, 2019 9:13 AM To: dev@kafka.apache.org Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi, I

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2019-01-14 Thread ChienHsing Wu
: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi guys, I am not sure what to do next in this KIP process. Could anyone please help/advise me on what to do next? Thanks, CH -Original Message- From: ChienHsing Wu Sent: Wednesday, January 02

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2019-01-07 Thread ChienHsing Wu
> -Original Message- > From: Mayuresh Gharat > Sent: Wednesday, December 12, 2018 12:55 PM > To: dev@kafka.apache.org > Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message > Consumption Across Partitions in KafkaConsumer > > Hi ChienHsing, > > We

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2019-01-02 Thread ChienHsing Wu
me on what to do next? > > Thanks! > > CH > > -Original Message- > From: ChienHsing Wu > Sent: Wednesday, December 12, 2018 1:05 PM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message > Consumption Across Partitions

Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2018-12-22 Thread Colin McCabe
ks! > > CH > > -Original Message- > From: ChienHsing Wu > Sent: Wednesday, December 12, 2018 1:05 PM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message > Consumption Across Partitions in KafkaConsumer > > Good to k

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2018-12-19 Thread ChienHsing Wu
: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Good to know that, Thanks! Nonetheless, that introduces additional complexity at the client side for a common expectation to more or less receives records in a fair fashion. CH -Original

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2018-12-12 Thread ChienHsing Wu
@kafka.apache.org Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi ChienHsing, We are actually working on buffering the already fetched data for paused topicPartitions, so ideally it should not have any effect on performance. Associated jira

Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2018-12-12 Thread Mayuresh Gharat
, B, B, B. > > > > Ideally, it should alternate between A and B. I was proposing to move > > to the next one fetch for the next poll call based on the order in the > > completed fetch queue, so the order becomes A, B, A, B, A, B. The > > implementation parses the comple

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2018-12-12 Thread ChienHsing Wu
the completed fetch earlier and have a list to manage them. CH -Original Message- From: Mayuresh Gharat Sent: Tuesday, December 11, 2018 6:58 PM To: dev@kafka.apache.org Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi

Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2018-12-11 Thread Mayuresh Gharat
ge- > From: Mayuresh Gharat > Sent: Tuesday, December 11, 2018 1:21 PM > To: dev@kafka.apache.org > Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption > Across Partitions in KafkaConsumer > > Hi ChienHsing, > > Thanks for the KIP. > It would b

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2018-12-11 Thread ChienHsing Wu
tRlgk=> > would be very valuable. > > > > Thanks, ChienHsing > > > > -Original Message- > From: ChienHsing Wu > Sent: Tuesday, December 04, 2018 11:43 AM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message > Consumption

Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2018-12-11 Thread Mayuresh Gharat
%3A+Fair+Message+Consumption+Across+Partitions+in+KafkaConsumer> > would be very valuable. > > > > Thanks, ChienHsing > > > > -Original Message- > From: ChienHsing Wu > Sent: Tuesday, December 04, 2018 11:43 AM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2018-12-11 Thread ChienHsing Wu
gt; Sent: Tuesday, November 20, 2018 2:46 PM To: dev@kafka.apache.org<mailto:dev@kafka.apache.org> Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi Matt, Thanks for the feedback. The issue with the current design is that

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2018-12-04 Thread ChienHsing Wu
-Original Message- From: Matt Farmer Sent: Monday, November 19, 2018 9:32 PM To: dev@kafka.apache.org Subject: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi there, Thanks for the KIP. We’ve run into issues with this at Mailchimp so something

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

2018-11-20 Thread ChienHsing Wu
Farmer Sent: Monday, November 19, 2018 9:32 PM To: dev@kafka.apache.org Subject: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi there, Thanks for the KIP. We’ve run into issues with this at Mailchimp so something to address consuming behavior