Re: Kafka 0.9 consumer API

2015-03-21 Thread Zijing Guo
Hi all,The document is very beautiful and the Kafka release version for this will be? and what is the timeline? ThanksEdwin On Friday, March 20, 2015 4:20 PM, Rajiv Kurian ra...@signalfuse.com wrote: Awesome - can't wait for this version to be out! On Fri, Mar 20, 2015 at 12:22

Re: Kafka DefaultPartitioner is not behaved as expected.

2015-03-05 Thread Zijing Guo
as the serializer.class. This StringEncoder takes string as input and as you know string always returns same hash code if the value is the same. On Fri, Mar 6, 2015 at 2:23 AM, Zijing Guo alter...@yahoo.com.invalid wrote: And also there something that I think worth mentioning,when I call prod.send

Kafka DefaultPartitioner is not behaved as expected.

2015-03-05 Thread Zijing Guo
Hi community,I have a 2 nodes test cluster with 2 zk instance and 2 broker instance running and I'm experimenting kafka producer in a cluster environment. So I create a topic foo with 2 partitions and replication 1.I create a async Producer without defining partition.class (so the partitioner

Re: Kafka DefaultPartitioner is not behaved as expected.

2015-03-05 Thread Zijing Guo
And I'm using kafka version 0.8.2.0 On Thursday, March 5, 2015 11:51 AM, Zijing Guo alter...@yahoo.com.INVALID wrote: Hi community,I have a 2 nodes test cluster with 2 zk instance and 2 broker instance running and I'm experimenting kafka producer in a cluster environment. So I

Re: Kafka DefaultPartitioner is not behaved as expected.

2015-03-05 Thread Zijing Guo
case key is test message + e. Thanks, Mayuresh On Thu, Mar 5, 2015 at 9:25 AM, Zijing Guo alter...@yahoo.com.invalid wrote: And I'm using kafka version 0.8.2.0       On Thursday, March 5, 2015 11:51 AM, Zijing Guo alter...@yahoo.com.INVALID wrote:   Hi community,I have a 2 nodes test

Re: Kafka DefaultPartitioner is not behaved as expected.

2015-03-05 Thread Zijing Guo
Hi Guozhang,I'm using kafka 0.8.2.0  Thanks On Thursday, March 5, 2015 12:57 PM, Guozhang Wang wangg...@gmail.com wrote: Zijing, Which version of Kafka client are you using? On Thu, Mar 5, 2015 at 8:50 AM, Zijing Guo alter...@yahoo.com.invalid wrote: Hi community,I have a 2 nodes

Re: Kafka DefaultPartitioner is not behaved as expected.

2015-03-05 Thread Zijing Guo
to bytes, the data is not going to the proper partitions. Thanks On Thursday, March 5, 2015 12:59 PM, Zijing Guo alter...@yahoo.com.INVALID wrote: Hi Guozhang,I'm using kafka 0.8.2.0  Thanks     On Thursday, March 5, 2015 12:57 PM, Guozhang Wang wangg...@gmail.com wrote

Re: understanding partition key

2015-02-11 Thread Zijing Guo
Partition key is on producer level, that if you have multiple partitions for a single topic, then you can pass in a key for the KeyedMessage object, and base on different partition.class, it will return a partition number for the producer, and producer will find the leader for that partition.I

Re: What ports does a Kafka Producer/Consumer use other than 2180's and 9090's?

2015-02-10 Thread Zijing Guo
The kafka's port can be configured through server.properties, and if you use kafka's zk, then you can modify the zookeeper.properties file, for clientPort. On Tuesday, February 10, 2015 2:11 PM, Su She suhsheka...@gmail.com wrote: I was looking at the documentation and I see that

Re: Can't send a keyedMessage to brokers with partitioner.class=kafka.producer.DefaultPartitioner

2015-02-09 Thread Zijing Guo
and zookeepers. And then I create a topic kafka-test with 2 partitions and replication-factor =2. My prod... | | | | View on stackoverflow.com | Preview by Yahoo | | | |   |   On Sunday, February 8, 2015 4:18 AM, Zijing Guo alter...@yahoo.com.INVALID wrote: Hi,I have a 2 nodes kafka

Can't send a keyedMessage to brokers with partitioner.class=kafka.producer.DefaultPartitioner

2015-02-08 Thread Zijing Guo
Hi,I have a 2 nodes kafka cluster with 2 instances of brokers and zookeepers. And then I create a topic kafka-test with 2 partitions and replication-factor =2. My producer config is:                      {partitioner.class kafka.producer.DefaultPartitioner                     

kafka consumers parallel consuming at consumer level or thread level?

2015-02-04 Thread Zijing Guo
Hi,I have some question regarding how kafka consumers achieve parallel consuming for one topic. Say I have 2 partitions for topic1 and I have a consumer Group A, now:1: If no consumer under consumer Group A subscribe topic1, then no message will be delivery to this consumer group.2: If there is

Apache Spark broadcast error: Error sending message as driverActor is null [message = UpdateBlockInfo(BlockManagerId

2015-01-22 Thread Zijing Guo
HiI'm using Apache Spark 1.1.0 and I'm currently having issue with broadcast method. So when I call broadcast function on a small dataset to a 5 nodes cluster, I experiencing the Error sending message as driverActor is null after broadcast the variables several times (apps running under jboss). 

Re: Apache Spark broadcast error: Error sending message as driverActor is null [message = UpdateBlockInfo(BlockManagerId

2015-01-22 Thread Zijing Guo
Yep, sorry, had a rough day.. On Thursday, January 22, 2015 2:25 PM, Guozhang Wang wangg...@gmail.com wrote: Hi Zijing, Sounds like you sent to the wrong mailing list :P Guozhang On Thu, Jan 22, 2015 at 11:12 AM, Zijing Guo alter...@yahoo.com.invalid wrote: HiI'm using Apache