Re: Regarding Kafka

Hans Thanks for the response, yeah you can say yeah I am treating topics like partitions, because my current logic of producing to a respective topic goes something like this RdKafka::ErrorCode resp = m_kafkaProducer->produce(m_kafkaTopic[whichTopic],

Re: Regarding Kafka

Why do you have 10 topics? It seems like you are treating topics like partitions and it's unclear why you don't just have 1 topic with 10, 20, or even 30 partitions. Ordering is only guaranteed at a partition level. In general if you want to capacity plan for partitions you benchmark a single

Re: client use high cpu which caused by delayedFetch operation immediately return

Can you check if you have KAFKA-3003 when you run the code? On Sat, Oct 8, 2016 at 12:52 AM, Kafka wrote: > Hi all, > we found our consumer have high cpu load in our product > enviroment，as we know，fetch.min.bytes and fetch.wait.ma < > http://fetch.wait.ma/>x.ms will affect the frequency

Re: Regarding Kafka

Guys any views ? Abhi On Sat, Oct 8, 2016 at 4:28 PM, Abhit Kalsotra wrote: > Hello > > I am using librdkafka c++ library for my application . > > *My Kafka Cluster Set up* > 2 Kafka Zookeper running on 2 different instances > 7 Kafka Brokers , 4 Running on 1 machine and 3 running on other mach

Re: Kafka streams Processor life cycle behavior of close()

Tnx! Looks like fix is already in for 0.10.1.0 On Tue, Oct 4, 2016 at 6:18 PM, Guozhang Wang wrote: > Created https://issues.apache.org/jira/browse/KAFKA-4253 for this issue. > > > Guozhang > > On Tue, Oct 4, 2016 at 3:08 PM, Guozhang Wang wrote: > > > Hello Srikanth, > > > > We close the under

Re: puncutuate() bug

Actually, I think the bug is more subtle. What happens when a consumed topic stops receiving messages? The smallest timestamp will always be the static timestamp of this topic. -David On 10/7/16, 5:03 PM, "David Garcia" wrote: Ok I found the bug. Basically, if there is an empty topic (

Re: Understanding org.apache.kafka.streams.errors.TopologyBuilderException

I don't think that is the issue. The join api says: public KTable join(KTable other, ValueJoiner joiner) In my case V is Map> V1 is List R is Map> K is String Note the final V and V1 are arrived after doing transformation on original streams So there are intermediate steps like stream.map(new K

RE: Understanding org.apache.kafka.streams.errors.TopologyBuilderException

> From: sjmit...@gmail.com > Date: Sat, 8 Oct 2016 15:30:33 +0530 > Subject: Understanding > org.apache.kafka.streams.errors.TopologyBuilderException > To: users@kafka.apache.org > > Hi, > I am get

Regarding Kafka

Hello I am using librdkafka c++ library for my application . *My Kafka Cluster Set up* 2 Kafka Zookeper running on 2 different instances 7 Kafka Brokers , 4 Running on 1 machine and 3 running on other machine Total 10 Topics and partition count is 3 with replication factor of 3. Now in my case I

Understanding org.apache.kafka.streams.errors.TopologyBuilderException

Hi, I am getting this exception Exception in thread "main" org.apache.kafka.streams.errors.TopologyBuilderException: Invalid topology building: KTABLE-MAPVALUES-07 and KSTREAM-AGGREGATE-09 are not joinable What I am trying to do is I aggregate a KStream into a KTable of type KTabl

client use high cpu which caused by delayedFetch operation immediately return

Hi all, we found our consumer have high cpu load in our product enviroment，as we know，fetch.min.bytes and fetch.wait.ma x.ms will affect the frequency of consumer’s return， so we adjust them to very big so that broker is very hard to satisfy it. then we foun