Re: Reg : Slowness in Kafka

2020-03-30 Thread Ismael Juma
Hi James, A JIRA would be helpful. It looks like something we should fix. Ismael On Mon, Mar 30, 2020 at 8:17 PM James Olsen wrote: > Christophe, > > See "Problems when Consuming from multiple Partitions” in the list > archive. I’ll forward you the full conversation privately. It includes >

Re: Reg : Slowness in Kafka

2020-03-30 Thread James Olsen
Christophe, See "Problems when Consuming from multiple Partitions” in the list archive. I’ll forward you the full conversation privately. It includes debug logs that demonstrate fetches being discarded and refetched, but only after one or more full expirations of the fetch.max.wait.ms even

Re: Problems when Consuming from multiple Partitions

2020-03-30 Thread James Olsen
Resolved by downgrading Client to 2.2.2 and implementing an application level heartbeat on every Producer to avoid he UNKNOWN_PRODUCER_ID issue. > On 9/03/2020, at 16:08, James Olsen wrote: > > P.S. I guess the big question is what is the best way to handle or avoid > UNKNOWN_PRODUCER_ID when

RE: Reg : Slowness in Kafka

2020-03-30 Thread JALADY Christophe
> There are serious latency issues when mixing different client and server > version Could you be more specific ? Link to any issue ? Thanks by advance ! Christophe De : James Olsen Envoyé : vendredi 27 mars 2020 01:48 À : users@kafka.apache.org Objet : Re:

Re: [kafka-clients] [VOTE] 2.5.0 RC2

2020-03-30 Thread Vahid Hashemian
Hi David, Thanks for running this release. Sorry for the delay in bringing this up. I just wanted to draw attention to https://issues.apache.org/jira/browse/KAFKA-9731 that blocked us from upgrading to 2.4. Based on the earlier discussion, the fix may not require a lot of work. Regards, --Vahid

Re: Are RocksDBWindowStore windows hopping or sliding?

2020-03-30 Thread Matthias J. Sax
`windowSize` has not impact on used segments: it is only used to compute the window end timestamp when you fetch a window, because the end-time is not stored explicitly (as all windows are assumed to have the same window size; i.e., it's a storage optimization to safe the bytes for window end

Brokers session terminated and removed

2020-03-30 Thread Pradeep Badiger
Hi, I am running into an issue where the Kafka brokers (0.10.2.1) are getting removed from the Zookeeper (3.4.14). Here is the setup. We have 3 Zookeeper nodes and 3 Kafka nodes in AWS. We are making use of auto-scaling group to get the replacement nodes on failures. When the Zookeeper and

Re: Are RocksDBWindowStore windows hopping or sliding?

2020-03-30 Thread Sachin Mittal
Hi, I understood how window stores are implemented using rocksdb. When creating an instance of RocksDBWindowStore we pass two additional arguments: retainDuplicates windowSize I have not clearly understood the purpose of these two. Like say in my application I just create one windowed store of a