Re: Partition ownership with high-level consumer

2015-10-07 Thread Ewen Cheslack-Postava
And you can get the current assignment in the new consumer after the rebalance completes too: https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L593 On Tue, Oct 6, 2015 at 5:27 PM, Gwen Shapira wrote: > Ah,

Kafka License Detail

2015-10-07 Thread sugumar analysis
Hi All, We are using Apache Kafka for our messaging functionalities. We have developed the messaging system using Kafka. Now we are planning to setup in production environment. *Please any one confirm me the Apache Kafka is free of cost?* *Can we use it in Production environment?* Thanks,

Re: Kafka License Detail

2015-10-07 Thread Karthikeyan Annamalai
Sugumar, Yes, It is free to use in your production environment as along you obey this. Btwn, I am wondering how did you missed this during your evolution. Good luck with your prod push. On Wed, Oct 7, 2015 at 3:21 PM, sugumar analysis

zk reconnection issues

2015-10-07 Thread John Firth
We've been having some problems with ZK session expiry resulting in either permanent disconnection of Kafka's ZK client, or sporadically successful reconnections followed closely by disconnections, which can produce OOMs if the reconnections occur close enough to one another. I've detailed the

Kafka dotnet SDK suggestion

2015-10-07 Thread sugumar analysis
Hi All, We are developing Messaging system with Kafka using DotNet (C#). We found there are 2 different SDK available for Dotnet 1. https://github.com/Jroland/kafka-net/ 2. https://github.com/ExactTargetDev/kafka-net First one is officially referred by Apache Kafka. But It is an initial

Re: log compaction scaling with ~100m messages

2015-10-07 Thread Joel Koshy
Using log compaction is well-suited for applications that use Kafka directly and need to persist some state associated with its processing. So something like offset management for consumers is a good fit. Another good use-case is for