Re: Flink vs Kafka streams

Thanks Peter, even with ECS we have autoscaling enabled but the issue is during autoscaling we need to stop the job and start with new parallelism which creates a downtime. Thanks On Fri, Nov 8, 2019 at 1:01 PM Peter Groesbeck wrote: > We use EMR instead of ECS but if that’s an option for your

Re: Subject: [VOTE] 2.2.2 RC2

+1 (non-binding) - Ran testAll/releaseTarGzAll successfully with no failures. - Ran through quickstart of core/streams on builds generated from 2.3.0-rc2 tag. - Ran a few internal apps with ~2500 topic-partitions on 5 a node cluster. Thanks, Satish. On Sat, Nov 9, 2019 at 4:41 AM Bill Bejeck

MirrorMaker 2 Plugin class loader Error

Hi Folks, I'm trying to run MM 2 with the current trunk. Also, tried with the 2.4 branch and I'm getting "ERROR Plugin class loader for connector: 'org.apache.kafka.connect.mirror.MirrorSourceConnector'" errors for all the connectors. It does not seem to be creating topics in the destination

Re: Subject: [VOTE] 2.2.2 RC2

- verified signatures - built it from source - ran unit tests +1 (non-binding) Thanks for running the release, Randall! -Bill On Fri, Nov 8, 2019 at 4:40 PM David Arthur wrote: > * Glanced through docs, release notes > * Downloaded RC2 binaries, verified signatures > * Ran through

Re: Subject: [VOTE] 2.2.2 RC2

* Glanced through docs, release notes * Downloaded RC2 binaries, verified signatures * Ran through quickstart +1 binding Thanks for managing this release, Randall! -David On Wed, Nov 6, 2019 at 7:39 PM Eric Lalonde wrote: > Hello, > > In an effort to assist in the verification of release

Re: Flink vs Kafka streams

We use EMR instead of ECS but if that’s an option for your team, you can configure auto scaling rules in your cloud formation so that your task/job load dynamically controls cluster sizing. Sent from my iPhone > On Nov 8, 2019, at 1:40 AM, Navneeth Krishnan > wrote: > > Hello All, > > I

Re: Message order and retries question

Hi, On Fri, 8 Nov 2019 at 17:19, Jose Manuel Vega Monroy < jose.mon...@williamhill.com> wrote: > Hi there, > > > > I have a question about message order and retries. > > > > After checking official documentation, and asking your feedback, we set > this kafka client configuration in each

Message order and retries question

Hi there, I have a question about message order and retries. After checking official documentation, and asking your feedback, we set this kafka client configuration in each producer: retries = 1 # note to ensure order enable.idempotence=true, which forcing to acks=all and

Re: [ANNOUNCE] New committer: Mickael Maison

Congrats Mickael!! :) On Fri, Nov 8, 2019 at 1:24 PM Satish Duggana wrote: > Congratulations Mickael!! > > On Fri, Nov 8, 2019 at 2:50 PM Rajini Sivaram > wrote: > > > > Congratulations, Mickael, well deserved!! > > > > Regards, > > > > Rajini > > > > On Fri, Nov 8, 2019 at 9:08 AM David Jacot

Re: [External] AW: Consumer Lags and receive no records anymore

A null key results in the client sending to partitions in a round-robin order. Use a key if you want to ensure that specific messages end up on the same partition. -Dave On 11/8/19, 1:06 AM, "Oliver Eckle" wrote: Hi, Don’t get me wrong, I just want to understand what's going on.

Re: [ANNOUNCE] New committer: Mickael Maison

Congratulations Mickael!! On Fri, Nov 8, 2019 at 2:50 PM Rajini Sivaram wrote: > > Congratulations, Mickael, well deserved!! > > Regards, > > Rajini > > On Fri, Nov 8, 2019 at 9:08 AM David Jacot wrote: > > > Congrats Mickeal, well deserved! > > > > On Fri, Nov 8, 2019 at 8:56 AM Tom Bentley

Re: Consumer Lags and receive no records anymore

Hi On Fri, 8 Nov 2019 at 07:06, Oliver Eckle wrote: > Hi, > > Don’t get me wrong, I just want to understand what's going on. > so how do I figure out, how much partitions are required? Trial and Error? > Normally, you have to run your perf test with adequate batching,partition,isr etc. to

Running Kafka Stream Application in YARN

Hi Is there any documentation or link I can refer to for the steps for deploying the Kafka Streams application in YARN? Kafka Client - 0.11.0.3 Kafka Broker - 2.2.1 YARN - 2.6.0

Re: [ANNOUNCE] New committer: Mickael Maison

Congratulations, Mickael, well deserved!! Regards, Rajini On Fri, Nov 8, 2019 at 9:08 AM David Jacot wrote: > Congrats Mickeal, well deserved! > > On Fri, Nov 8, 2019 at 8:56 AM Tom Bentley wrote: > > > Congratulations Mickael! > > > > On Fri, Nov 8, 2019 at 6:41 AM Vahid Hashemian < >

Re: [ANNOUNCE] New committer: Mickael Maison

Congrats Mickeal, well deserved! On Fri, Nov 8, 2019 at 8:56 AM Tom Bentley wrote: > Congratulations Mickael! > > On Fri, Nov 8, 2019 at 6:41 AM Vahid Hashemian > wrote: > > > Congrats Mickael, > > > > Well deserved! > > > > --Vahid > > > > On Thu, Nov 7, 2019 at 9:10 PM Maulin Vasavada < >