Thank you Erik.
In my test I am using fixed 200bytes messages and I run 500k messages per
producer on 92 physically isolated producers. Each test run takes about 20
minutes. As the broker cluster is migrated into a new physical cluster, I
will perform my test and get the latency results in the nex
Yes, and that can really hurt average performance. All the partitions
were nearly identical up to the 99%’ile, and had very good performance at
that level hovering around a few milli’s. But when looking beyond the
99%’ile, there was that clear fork in the distribution where a set of 3
partitions
So are you suggesting that the long delays happened in %1 percentile
happens in the slower partitions that are further away? Thanks.
On Wed, Sep 9, 2015 at 3:15 PM, Helleren, Erik
wrote:
> So, I did my own latency test on a cluster of 3 nodes, and there is a
> significant difference around the 9
So, I did my own latency test on a cluster of 3 nodes, and there is a
significant difference around the 99%’ile and higher for partitions when
measuring the the ack time when configured for a single ack. The graph
that I wish I could attach or post clearly shows that around 1/3 of the
partitions s
No problem. Thanks for your advice. I think it would be fun to explore. I
only know how to program in java though. Hope it will work.
On Fri, Sep 4, 2015 at 2:03 PM, Helleren, Erik
wrote:
> I thing the suggestion is to have partitions/brokers >=1, so 32 should be
> enough.
>
> As for latency tes
I thing the suggestion is to have partitions/brokers >=1, so 32 should be
enough.
As for latency tests, there isn’t a lot of code to do a latency test. If
you just want to measure ack time its around 100 lines. I will try to
push out some good latency testing code to github, but my company is
Thanks for your reply Erik. I am running some more tests according to your
suggestions now and I will share with my results here. Is it necessary to
use a fixed number of partitions (32 partitions maybe) for my test?
I am testing 2, 4, 8, 16 and 32 brokers scenarios, all of them are running
on ind
That is an excellent question! There are a bunch of ways to monitor
jitter and see when that is happening. Here are a few:
- You could slice the histogram every few seconds, save it out with a
timestamp, and then look at how they compare. This would be mostly
manual, or you can graph line chart
Thank you Erik! That's is helpful!
But also I see jitters of the maximum latencies when running the
experiment.
The average end to acknowledgement latency from producer to broker is
around 5ms when using 92 producers and 4 brokers, and the 99.9 percentile
latency is 58ms, but the maximum latency
WellŠ not to be contrarian, but latency depends much more on the latency
between the producer and the broker that is the leader for the partition
you are publishing to. At least when your brokers are not saturated with
messages, and acks are set to 1. If acks are set to ALL, latency on an
non-sat
When I using 32 partitions, the 4 brokers latency becomes larger than the 8
brokers latency.
So is it always true that using more brokers can give less latency when the
number of partitions is at least the size of the brokers?
Thanks.
On Thu, Sep 3, 2015 at 10:45 PM, Yuheng Du wrote:
> I am ru
Thank you, Tao!
On Jul 15, 2015 6:27 PM, "Tao Feng" wrote:
> Sorry Yufeng, You should change it in $KAFKA_HEAP_OPTS.
>
> On Wed, Jul 15, 2015 at 3:09 PM, Tao Feng wrote:
>
> > Hi Yuheng,
> >
> > You could add the -Xmx1024m in
> > https://github.com/apache/kafka/blob/trunk/bin/kafka-run-class.sh
Sorry Yufeng, You should change it in $KAFKA_HEAP_OPTS.
On Wed, Jul 15, 2015 at 3:09 PM, Tao Feng wrote:
> Hi Yuheng,
>
> You could add the -Xmx1024m in
> https://github.com/apache/kafka/blob/trunk/bin/kafka-run-class.sh
> KAFKA_JVM_PERFORMANCE_OPTS.
>
>
>
> On Wed, Jul 15, 2015 at 12:51 AM, Yu
Hi Yuheng,
You could add the -Xmx1024m in
https://github.com/apache/kafka/blob/trunk/bin/kafka-run-class.sh
KAFKA_JVM_PERFORMANCE_OPTS.
On Wed, Jul 15, 2015 at 12:51 AM, Yuheng Du
wrote:
> Tao,
>
> If I am running on the command line the following command
> >bin/kafka-run-class.sh kafka.tools
I have run the end to end latency test and the producerPerformance test on
my kafka cluster according to
https://gist.github.com/jkreps/c7ddb4041ef62a900e6c
In end to end latency test, the latency was around 2ms. In
producerperformance test, if use batch size 8196 to send 50,000,000 records:
>bin
Tao,
If I am running on the command line the following command
>bin/kafka-run-class.sh kafka.tools.TestEndToEndLatency 192.168.1.3:9092
192.168.1.1:2181 speedx3 5000 100 1 -Xmx1024m
It promped that it is not correct. So where should I put the -Xmx1024m
option? Thanks.
On Wed, Jul 15, 2015
(Please correct me if I am wrong.) Based on TestEndToEndLatency(
https://github.com/apache/kafka/blob/trunk/core/src/test/scala/kafka/tools/TestEndToEndLatency.scala),
consumer_fetch_max_wait corresponds to fetch.wait.max.ms in consumer config(
http://kafka.apache.org/documentation.html#consumerco
I got java out of heap error when running end to end latency test:
yuhengdu@consumer0:/packages/kafka_2.10-0.8.2.1$ bin/kafka-run-class.sh
kafka.tools.TestEndToEndLatency 192.168.1.3:9092 192.168.1.1:2181 speedx3
5000 100 1
Exception in thread "main" java.lang.OutOfMemoryError: Java heap spac
Tao,
Thanks. The example on https://gist.github.com/jkreps/c7ddb4041ef62a900e6c
is outdated already. The error message shows:
USAGE: java kafka.tools.TestEndToEndLatency$ broker_list zookeeper_connect
topic num_messages consumer_fetch_max_wait producer_acks
Can anyone helps me what should be pu
I think ProducerPerformance microbenchmark only measure between client to
brokers(producer to brokers) and provide latency information.
On Tue, Jul 14, 2015 at 11:05 AM, Yuheng Du
wrote:
> Currently, the latency test from kafka test the end to end latency between
> producers and consumers.
>
>
20 matches
Mail list logo
