[ANN] kafka_ex (Kafka Client Library for Elixir)

Hi all, I am announcing a Kafka client library for Elixir that I co-authored, you can find the library at: https://github.com/jacklund/kafka_ex A blog post explaining how to use the library can be found at: http://www.abeji.de/kafka-meets-elixir/ Full API documentation is at : http://hexdoc

RE: Consumer in Java client

2 more questions to add to it: - Any estimation for when new consumer client will be ready for high level consumer? - Also, will it come with offset management functionality for high level consumer too? -Original Message- From: Jiangjie Qin [mailto:j...@linkedin.com.INVALID] Sent: Mond

Re: Kafka 8.2.1 Offset fetch Request

On a first pass, the code has a small bug in getConsumedOffset(). You need to close the channel after getting the coordinator : // broker = cmr.coordinator(); and then reconnect to coordinator as follows : broker = cmr.coordinator(); // if the coordinator is different, from the above cha

Re: schemaregistry example

The name for the int type in Avro is "int" not "integer". Your command should work if you change field2's type. -Ewen On Tue, Mar 31, 2015 at 1:51 AM, Clint Mcneil wrote: > Hi guys > > When trying the example schema in > http://confluent.io/docs/current/schema-registry/docs/api.html > > POST /s

Re: Can Mirroring Preserve Every Topic's Partition?

The Mirror maker in trunk now supports mirroring with preserved partition. You can wire in a message handler to assign partitions for each producer record before handing them to producer. Jiangjie (Becket) Qin On 3/31/15, 3:41 AM, "Ivan Balashov" wrote: >Alex, > >Just wondering, did you have an

Re: blocking KafkaProducer call

thanks ghenke, that was a quick response. I will test and will let you know if i have some questions. On Tue, Mar 31, 2015 at 11:17 PM, Grant Henke wrote: > I think you are looking at is this section: > > > If you want to simulate a simple blocking call you can do the following: > > > > producer

Re: blocking KafkaProducer call

I think you are looking at is this section: > If you want to simulate a simple blocking call you can do the following: > > producer.send(new ProducerRecord("the-topic", > "key".getBytes(), "value".getBytes())).get(); > > What that is doing is calling .get() on the Future returned by the send meth

blocking KafkaProducer call

Hi According to this http://kafka.apache.org/082/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html documentation, if i don't pass callback it will work as blocking call, Does it mean that message will be immediately sent to kafka cluster and all possible exceptions will be thr

Re: Kafka server relocation

Can you attach your mirror maker log? On Wed, Apr 1, 2015 at 12:28 AM, nitin sharma wrote: > i tried with auto.offset.reset=smallest, but still not working.. > > there is data in my source cluster > > Regards, > Nitin Kumar Sharma. > > > On Mon, Mar 30, 2015 at 10:30 PM, tao xiao wrote: > > > D

Re: Kafka server relocation

i tried with auto.offset.reset=smallest, but still not working.. there is data in my source cluster Regards, Nitin Kumar Sharma. On Mon, Mar 30, 2015 at 10:30 PM, tao xiao wrote: > Do you have data sending to *testtopic? *By default mirror maker only > consumes data being sent after it taps i

Re: Kafka Consumer

Many thanks for the responses. Harsh your response has clarified things a bit for me. I'm trying from Spark: KafkaUtils.createStream(spark, "zkHos:2181", "groupid", topics); And wasn't sure what value i should use for 'group' in consumer docs it says default is groupid so used that. But will r

Re: Kafka Consumer

Hi James,         Can you elaborate on what you mean by group here?. There are no groups on the topic side but there are consumer groups and these will be on consumer side . "Consumers label themselves with a consumer group name, and each message published to a topic is delivered to one consumer

Re: Kafka Consumer

Your consumer "might" belong to a consumer group. Just commit offsets to that consumer groups/topic/partition and it will work. That said - if you want to figure out the consumers groups that exists you have to look in zookeeper. There is no kafka API to get or create them. In the java client it i

Kafka Consumer

I created a topic using: bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test How do I find out what group it belongs to? Thank you.

Re: Can Mirroring Preserve Every Topic's Partition?

Alex, Just wondering, did you have any success patching and running MM with exact partitioning support? If so, could you possibly share the patch and, as I hope, your positive experience with the process? Thanks!

schemaregistry example

Hi guys When trying the example schema in http://confluent.io/docs/current/schema-registry/docs/api.html POST /subjects/test HTTP/1.1Host: schemaregistry.example.comAccept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json { "schema": "{