Re: 答复: kafka performance question

2014-05-26 Thread svante karlsson
per block. > > > -邮件原件- > 发件人: robairrob...@gmail.com [mailto:robairrob...@gmail.com] 代表 Robert > Turner > 发送时间: 2014年5月16日 13:45 > 收件人: users@kafka.apache.org > 主题: Re: kafka performance question > > A couple of thoughts spring to mind, are you sending the wh

答复: kafka performance question

2014-05-25 Thread Zhujie (zhujie, Smartcare)
: Re: kafka performance question A couple of thoughts spring to mind, are you sending the whole file as 1 message? and is your producer code using sync or async mode? Cheers Rob. On 14 May 2014 15:49, Jun Rao wrote: > How many brokers and partitions do you have? You may try increas

Re: kafka performance question

2014-05-16 Thread Robert Turner
A couple of thoughts spring to mind, are you sending the whole file as 1 message? and is your producer code using sync or async mode? Cheers Rob. On 14 May 2014 15:49, Jun Rao wrote: > How many brokers and partitions do you have? You may try increasing > batch.num.messages. > > Thanks, > >

Re: kafka performance question

2014-05-15 Thread Jun Rao
How many brokers and partitions do you have? You may try increasing batch.num.messages. Thanks, Jun On Tue, May 13, 2014 at 5:56 PM, Zhujie (zhujie, Smartcare) < first.zhu...@huawei.com> wrote: > Dear all, > > We want to use kafka to collect and dispatch data file, but the > performance is may

kafka performance question

2014-05-13 Thread Zhujie (zhujie, Smartcare)
our version is kafka_2.10-0.8.1 发件人: Zhujie (zhujie, Smartcare) 发送时间: 2014年5月14日 8:56 收件人: 'users@kafka.apache.org'; 'd...@kafka.apache.org' 主题: kafka performance question Dear all, We want to use kafka to collect and dispatch data file, but the performance is maybe lower t

kafka performance question

2014-05-13 Thread Zhujie (zhujie, Smartcare)
Dear all, We want to use kafka to collect and dispatch data file, but the performance is maybe lower than we want. In our cluster,there is a provider and a broker. We use a one thread read file from local disk of provider and send it to broker. The average throughput is only 3 MB/S~4MB/S. But