java.net.SocketTimeoutException in Broker

Hi All, This is what we are running on the Broker: /Users/worun/kafkabuild/kafka-0.8.1-src/bin/kafka-producer-perf-test.sh --broker-list vp21q12ic-hpaj020921:9092 --messages 1000 --topic imessage --threads 10 --message-size 420 --batch-size 1000 --compression-codec 1 We are getting the b

答复: how many instances of the broker is suggested in one server.

Is there any principles? -邮件原件- 发件人: Jun Rao [mailto:jun...@gmail.com] 发送时间: 2014年5月27日 11:25 收件人: users@kafka.apache.org 主题: Re: how many instances of the broker is suggested in one server. The number of instances required will depend on the load. However, keep in mind that all instan

HA: multi-broker topic rebalance

Hello! >>Which version of Kafka are you using? 1. I am using 0.8.1 >>How many partitions do you have in that topic? 2. In docs I found that number of partition must be equal or greater than number of consumers ... so I try 2 case: - 10k partition per broker - 20k partition per broker And

Re: kafka broker failed to recovery from ZK failure

well. I just run the kafka-run-class.sh script with DeleteTopicCommand class directly. I noticed the warning on documentation page. but I thought it is only about not complete cleanup. I didn't know it can affect cluster health. maybe it's worthwhile to point it out the

Re: multi-broker topic rebalance

Which version of Kafka are you using? How many partitions do you have in that topic? Thanks, Jun On Mon, May 26, 2014 at 7:14 AM, Рябков Алексей Николаевич < a.ryab...@ntc-vulkan.ru> wrote: > > Hello! > > How can I rebalance topic within multiple brokers? > For example: > > 1.I have ab

Re: how many instances of the broker is suggested in one server.

The number of instances required will depend on the load. However, keep in mind that all instances on a server will compete for memory (e.g., for pagecache, jvm heap, etc), storage (log space) and CPU. Thanks, Jun On Sun, May 25, 2014 at 11:10 PM, Zhujie (zhujie, Smartcare) < first.zhu...@huawe

Re: Unable to scale kafka cluster

To add partitions, you need to use the --alter option in kafka-topics.sh. Thanks, Jun On Sat, May 24, 2014 at 3:52 AM, Ertuğrul Yılmaz < ertugrul.yil...@convertale.com> wrote: > Hi, > > I know replica partition tools. I try to add new partition and assign to a > new broker during producer test

multi-broker topic rebalance

Hello! How can I rebalance topic within multiple brokers? For example: 1.I have about 30k consumers... 2.I need to use 3 broker (10k consumer per broker) 3. I use multiple (around 10) producers with default balance strategy Sometimes one broker have empty topic (consumer

multi-broker topic rebalance

Hello! How can I rebalance topic within multiple brokers? For example: 1.I have about 30k consumers... 2.I need to use 3 broker (10k consumer per broker) 3. I use multiple (around 10) producers with default balance strategy Sometimes one broker have empty topic (consumer

Re: Mbean "kafka.server":type="BrokerTopicMetrics",name="AllTopicsMessagesInPerSec" returning 0 always

Hi Arjun , What is the configured number of logical partitions per topic per server . With the partition size as 2 , Only two partitions would be created distributed over 2 brokers with one partition each , you can confirm the same by checking the log file directory for each broker (default - /tm

Mbean "kafka.server":type="BrokerTopicMetrics",name="AllTopicsMessagesInPerSec" returning 0 always

Hi, We have a Kafka cluster which contains 3 kafka brokers. Data is pushed into kafka queue continously. To monitor the kafka brokers, we use the JMX metrics given by kafka. One thing i observed is there is one broker which is returning*"0"* all the time for the bean *"kafka.server":type="Brok

Re: 答复: kafka performance question

Do you read from the file in the callback from kafka? I just implemented c++ bindings and in one of the tests i did I got the following results: 1000 messages per batch (fairly small messages ~150 bytes) and then wait for the network layer to ack the send (not server ack)'s before putting another