Re: New producer: metadata update problem on 2 Node cluster.

Creating a new consumer instance *does not* solve this problem. Attaching the producer/consumer code that I used for testing. On Wed, May 6, 2015 at 6:31 AM, Ewen Cheslack-Postava e...@confluent.io wrote: I'm not sure about the old producer behavior in this same failure scenario, but

Re: New producer: metadata update problem on 2 Node cluster.

Sorry, I meant creating a new producer, not consumer. Here's the code. Producer - http://pastebin.com/Kqq1ymCX Consumer - http://pastebin.com/i2Z8PTYB Callback - http://pastebin.com/x253z7bG As you'll notice, I am creating a new producer for each message. So the bootstrap nodes should be

RE: Significance of SimpleConsumer id string

A client id is used to logically identify an application. Ideally, multiple consumers belonging to the same application should use the same client id. More concretely, metrics can be gathered per client and quotas in the future will be enforced by clientId. Aditya

Re: New producer: metadata update problem on 2 Node cluster.

Rahul, the mailing list filters attachments, you'd have to post the code somewhere else for people to be able to see it. But I don't think anyone suggested that creating a new consumer would fix anything. Creating a new producer *and discarding the old one* basically just makes it start from

Auto-rebalance not triggering in 2.10-0.8.1.1

I'm running 2.10-0.8.1.1, and rebalance will not trigger on it's own. From http://grokbase.com/t/kafka/users/14bj5ps9hp/partition-auto-rebalance#20141118rf39q8cs4sjh6vzjgdw92e37cw I think the leader imbalance means: For a single broker, add up all the partitions it is leading (Y), and count the

Custom Partition example in SCALA

Hi, Could you please point me the example scala program for Custom Partition Regards, Rajesh

Getting NotLeaderForPartitionException in kafka broker

Hi team, I have a 12 nodes cluster that has 800 topics and each of which has only 1 partition. I observed that one of the node keeps generating NotLeaderForPartitionException that causes the node to be unresponsive to all requests. Below is the exception [2015-05-07 04:16:01,014] ERROR

Significance of SimpleConsumer id string

Hi, The kafka.consumer.SimpleConsumer takes an id: String” constructor parameter. What is the significance of this id? Are there any consequences or risks associated with using the exact same id for several consumers of the same topic/partition? /Magnus

Differences between new and legacy scala producer API

Hi - Legacy scala api for producer is having keyed message along with topic, key, partkey, and message. Meanwhile new api has no partkey. Whats the difference between key and partkey? - In javadoc, new producer api send method is always async, does producer.type properties overriden? - Will scala

Re: Support https or ssl

Hi Jamie,         I am currently working on providing ssl support for kafka. Here are the iras https://issues.apache.org/jira/browse/KAFKA-1690 and  https://issues.apache.org/jira/browse/KAFKA-1684 . If you are using REST api to front kafka producer than you can probably make that http server to

Support https or ssl

Hello, It's been a while since my team worked on kafka related project. Btw, previous project using Kafka worked wonderfully for us. Now I have requirement to use https or SSL. I am wondering if the latest version has support for SSL. If not, what is the timeline this functionality would

Re: Custom Partition example in SCALA

Hi, I have written below custom partition program in scala but it is not calling partition method from producer *class TestRoundRobinPartitioner(props: VerifiableProperties) extends Partitioner { def partition(key: Any, numPartitions: Int): Int = { println(key + key);