Re: Kafka ConsumerRebalanceFailedException, kafka.consumer.SimpleConsumer: Reconnect due to socket error: null, etc.

2015-05-26 Thread Jaikiran Pai
One way to narrow down the issue is to attach a debugger to the Kafka JVM and add a breakpoint in SimpleConsumer to see the real exception stacktrace which is causing the reconnect. I've filed a JIRA with a patch to improve this logging to include the entire cause stacktrace while logging this

Re: Is it a good idea to use Apache Kafka for this purpose?

2015-05-26 Thread Warren Henning
There's not much specific detail in what you described, but that sounds like classic pub-sub, which Kafka is well-suited for. On Tue, May 26, 2015 at 8:09 AM, Mert metin mertme...@outlook.com wrote: Hi there! I have multiple servers. What I want to do is to send user information across

Ordered Message Queue with Pool of Consumers

2015-05-26 Thread Kumar Jayanti
Our system has a specific message type that requires total ordering. Message-1 should be consumed (acted upon) before Message-2 is consumed. However since we have a cluster of consumer nodes we would like to round-robin the processing of messages if possible to different nodes while maintaining

Kafka Not Commiting Messages

2015-05-26 Thread Charlie Mason
Hi All, I have been trying to get started with Kafka. I have set up an 0.8.2 broker as per the quick start. With a single node broker I am able to run the scripts in the bin folder to successfully produce and consume messages. I then tried to write some Scala code to use the new 0.8 Producer API

Is fetching from in-sync replicas possible?

2015-05-26 Thread Theo Hultberg
Hello, Apologies if this question has been asked before. If I understand things correctly a client can only fetch from the leader of a partition, not from an (in-sync) replica. I have a use case where it would be very beneficial if it were possible to fetch from a replica instead of just the

Re: Is fetching from in-sync replicas possible?

2015-05-26 Thread Joel Koshy
Apologies if this question has been asked before. If I understand things correctly a client can only fetch from the leader of a partition, not from an (in-sync) replica. I have a use case where it would be very beneficial if it were possible to fetch from a replica instead of just the leader,

Re: Need some guidance in how to solve this problem.

2015-05-26 Thread Guozhang Wang
Hmm, that would be a problem especially if your WebSocket clients can come and go. I am actually wondering if your WebSocket is needed for any other usage beside this? If not, then you may probably get rid of the server and just let all clients directly consume from Kafka, where each one of them

Re: Log file of server start up error

2015-05-26 Thread Sriharsha Chintalapani
       It looks like you have another process running for kafka broker. Stop that broker and start a new one. --  Harsha On May 24, 2015 at 11:05:27 PM, Sanjay Mistry (mistrysanja...@gmail.com) wrote: Hello, When run this command bin/kafka-console-producer.sh --broker-list localhost:9092

exception connecting to Kafka when IPv6 textual representation of the address is returned

2015-05-26 Thread Marina
Hi, I keep running into the issue already discussed here:https://mail-archives.apache.org/mod_mbox/kafka-dev/201503.mbox/%3cjira.12718028.1401754422000.135548.1426712079...@atlassian.jira%3E where I am getting the javax.management.MalformedObjectNameException: Invalid character ':' in value

Re: Ordered Message Queue with Pool of Consumers

2015-05-26 Thread Kumar Jayanti
Thanks for your response. The requirement is to have total ordering across partitions. I do have a Key and all my consumers would be in the same consumer group. I am new to kafka and so would like to reconfirm my understanding of your last statement : So assuming you have a key you can

Is it a good idea to use Apache Kafka for this purpose?

2015-05-26 Thread Mert metin
Hi there! I have multiple servers. What I want to do is to send user information across servers. Basically, there will be publisher servers (masters) and subscribers(slaves). Each subscriber server will have a scope and it will only accept the information from that scope. I thought I can use

Re: How Producer handles Network Connectivity Issues

2015-05-26 Thread Ewen Cheslack-Postava
It's not being switched in this case because the broker hasn't failed. It can still connect to all the other brokers and zookeeper. The only failure is of the link between a client and the broker. Another way to think of this is to extend the scenario with more producers. If I have 100 other