No replica allocated to new Topic

Hi We have an issue where a Topic is created (auto create) but no partition replicas are allocated. This issue has been replicated on 0.9.0.1 and 0.8.2.2 The environment have multiple concurrent consumers and producers accessing the Kafka cluster and when we create topics they randomly but

Re: kafka client and streams jars for RC builds

Hi Jeff, We do not have kafka streams jar in maven since 0.10.0.0 is not released yet. For now you need to build them yourself. We are pushing hardly to have 0.10.0.0 out asap. Guozhang On Thu, Apr 7, 2016 at 6:56 AM, Jeff Klukas wrote: > I'm developing a Kafka Streams

increasing lag issue

Hello, I'm looking for guidance for a lag issue with very low level incoming traffic, that turns to an increasing lag (10K) in rush hour. Curiously I'm not getting any part of my HW not nearly 60% used. I've drawn the architecture so you can inspect the HW and Configuration details. *Some steps I

Re: [0.9.0.1] Having problems with commitAsync() not updating offsets

Hmm... gist didn't come thru for some reason.Trying agin: gist.github.com/gzoller/93fe2392fd3606bcb3b879e4ab2f8f6e From: Greg Zoller To: "users@kafka.apache.org" Sent: Thursday, April 7, 2016 5:25 PM Subject: [0.9.0.1] Having

[0.9.0.1] Having problems with commitAsync() not updating offsets

Hello, I'm having trouble with commitAsync() not updating my partition offsets.  I pre-populate 1,000,000 strings on topic 'lowercaseStrings' across 4 partitions.  I check the offsets and they're fine--250K records/partition. Then I run some code that uses a single-thread with blocking queue to

Re: Kafka Connect concept question

Another way to think about this is that the producer allows you to PUSH data into Kafka and the consumer allows you to PULL data out. This is what you need to write an application. However for an existing data system you need the opposite you need to PULL data into Kafka from the system or PUSH

Re: Kafka Connect concept question

On Wed, Apr 6, 2016 at 8:56 AM, Uber Slacker wrote: > Hi folks. I'm pretty new to Kafka. I have spent a fair amount of time so > far understanding the Kafka system in general and how producers and > consumers work. I'm now trying to get a grasp on how Kafka Connect >

MirrorMaker Status

Hello! Over half a year ago I noted that the MirrorMaker just stops after hitting a corrupt message, since no error handling is built in. Since some time has passed now I wanted to ask you: what do you use for mirroring now? Which issues have you had? Thanks Jörg

kafka client and streams jars for RC builds

I'm developing a Kafka Streams application and trying to keep up with the evolution of the API as 0.10.0.0 release candidates come out. We've got a test cluster running RC0, but it doesn't look like client or streams jars for the RCs are available on maven central. Are there any plans to upload

Issue of kafka 0.8.2.1 scala consumer

Hi, I encountered a issue when using kafka 0.8.2.1 scala consumer. The program cannot exit normally since it was blocking on committing the offset to broker (the offset storage is "kafka"). When committing offset to offset manager, the method

RE: KafkaProducer block on send

I totally agree with Oleg. As documentation says the producers send data in an asynchronous way and it is enforced by the send method signature with a Future returned. It can't block indefinitely without returning to the caller. I'm mean, you can decide that the code inside the send method

KafkaProducer block on send

I know it’s been discussed before, but that conversation never really concluded with any reasonable explanation, so I am bringing it up again as I believe this is a bug that would need to be fixed in some future release. Can someone please explain the rational for the following code in