Re: Consumer in Java client

Hi, KafkaConsumer is still under development and not ready for wide use case. Currently, it can be used to replace SimpleConsumer (low level consumer), but can not replace ZookeeperConsumerConnector(high level consumer). So if you need to use simple consumer, I would suggest using KafkaConsumer

Re: Async producer using Sync producer for send

The async send() put the message into a message queue then returns. When the messages are pulled out of the queue by the sender thread, it still uses SyncProducer to send ProducerRequests to brokers. Jiangjie (Becket) Qin On 3/30/15, 10:44 PM, Madhukar Bharti bhartimadhu...@gmail.com wrote: Hi

Re: Async producer using Sync producer for send

Thanks Jiangjie, I too have thought the same after looking the code. Thanks a lot for clearing my doubt! On Tue, Mar 31, 2015 at 11:45 AM, Jiangjie Qin j...@linkedin.com.invalid wrote: The async send() put the message into a message queue then returns. When the messages are pulled out of the

Kafka Consumer

I created a topic using: bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test How do I find out what group it belongs to? Thank you.

[ANN] kafka_ex (Kafka Client Library for Elixir)

Hi all, I am announcing a Kafka client library for Elixir that I co-authored, you can find the library at: https://github.com/jacklund/kafka_ex A blog post explaining how to use the library can be found at: http://www.abeji.de/kafka-meets-elixir/ Full API documentation is at :

Re: Kafka Consumer

Hi James,         Can you elaborate on what you mean by group here?. There are no groups on the topic side but there are consumer groups and these will be on consumer side . Consumers label themselves with a consumer group name, and each message published to a topic is delivered to one consumer

Re: Kafka server relocation

Can you attach your mirror maker log? On Wed, Apr 1, 2015 at 12:28 AM, nitin sharma kumarsharma.ni...@gmail.com wrote: i tried with auto.offset.reset=smallest, but still not working.. there is data in my source cluster Regards, Nitin Kumar Sharma. On Mon, Mar 30, 2015 at 10:30 PM, tao

Re: schemaregistry example

The name for the int type in Avro is int not integer. Your command should work if you change field2's type. -Ewen On Tue, Mar 31, 2015 at 1:51 AM, Clint Mcneil clintmcn...@gmail.com wrote: Hi guys When trying the example schema in

blocking KafkaProducer call

Hi According to this http://kafka.apache.org/082/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html documentation, if i don't pass callback it will work as blocking call, Does it mean that message will be immediately sent to kafka cluster and all possible exceptions will be

Re: blocking KafkaProducer call

I think you are looking at is this section: If you want to simulate a simple blocking call you can do the following: producer.send(new ProducerRecordbyte[],byte[](the-topic, key.getBytes(), value.getBytes())).get(); What that is doing is calling .get() on the Future returned by the send

Re: blocking KafkaProducer call

thanks ghenke, that was a quick response. I will test and will let you know if i have some questions. On Tue, Mar 31, 2015 at 11:17 PM, Grant Henke ghe...@cloudera.com wrote: I think you are looking at is this section: If you want to simulate a simple blocking call you can do the following:

Re: Can Mirroring Preserve Every Topic's Partition?

The Mirror maker in trunk now supports mirroring with preserved partition. You can wire in a message handler to assign partitions for each producer record before handing them to producer. Jiangjie (Becket) Qin On 3/31/15, 3:41 AM, Ivan Balashov ibalas...@gmail.com wrote: Alex, Just wondering,

Re: Kafka Consumer

Your consumer might belong to a consumer group. Just commit offsets to that consumer groups/topic/partition and it will work. That said - if you want to figure out the consumers groups that exists you have to look in zookeeper. There is no kafka API to get or create them. In the java client it is

RE: Consumer in Java client

2 more questions to add to it: - Any estimation for when new consumer client will be ready for high level consumer? - Also, will it come with offset management functionality for high level consumer too? -Original Message- From: Jiangjie Qin [mailto:j...@linkedin.com.INVALID] Sent:

Re: Kafka 8.2.1 Offset fetch Request

On a first pass, the code has a small bug in getConsumedOffset(). You need to close the channel after getting the coordinator : // broker = cmr.coordinator(); and then reconnect to coordinator as follows : broker = cmr.coordinator(); // if the coordinator is different, from the above