Implementing a custom partitioner

2015-07-21 Thread JIEFU GONG
Hi all, If I wanted to write my own partitioner, all I would need to do is write a class that extends Partitioner and override the partition function, correct? I am currently doing so, at least, with a class in the package 'services', yet when I use: properties.put(partitioner.class,

Re: Implementing a custom partitioner

2015-07-21 Thread Sriharsha Chintalapani
Hi,      Are you using the latest trunk for Producer API?.  Did you implement the interface here  https://cwiki.apache.org/confluence/display/KAFKA/KIP-+22+-+Expose+a+Partitioner+interface+in+the+new+producer --  Harsha On July 21, 2015 at 2:27:05 PM, JIEFU GONG (jg...@berkeley.edu) wrote: Hi

Re: Implementing a custom partitioner

2015-07-21 Thread JIEFU GONG
Sriharsha, thanks for your response. I'm using version 0.8.2, and I am implementing kafka.producer.Partitioner. I noticed that in the latest trunk the line I specified above is replaced with: this.partitioner = config.getConfiguredInstance(ProducerConfig. PARTITIONER_CLASS_CONFIG,

Re: Implementing a custom partitioner

2015-07-21 Thread Sriharsha Chintalapani
If you are using the new producer api from kafka 0.8.2 there is no pluggable partitioner in it for this you need to use the latest trunk. But in 0.8.2 if you are using old producer code you can implement a pluggable partitioner