Jiefu,
Have you tried to run benchmark_test.py? I ran it and it asks me for the
ducktape.services.service
yuhengdu@consumer0:/packages/kafka_2.10-0.8.2.1$ python benchmark_test.py
Traceback (most recent call last):
File benchmark_test.py, line 16, in module
from
Hi Geoffrey,
Thank you for your helpful information. Do I have to install the virtual
machines? I am using Mac as the testdriver machine or I can use a linux
machine to run testdriver too.
Thanks.
best,
Yuheng
On Wed, Jul 15, 2015 at 2:55 PM, Geoffrey Anderson ge...@confluent.io
wrote:
Hi
Hi Yuheng,
Running these tests requires a tool we've created at Confluent called
'ducktape', which you need to install with the command:
pip install ducktape==0.2.0
Running the tests locally requires some setup (creation of virtual machines
etc.) which is outlined here:
Hi Yuheng,
Yes, you should be able to run on either mac or linux.
The test cluster consists of a test-driver machine and some number of slave
machines. Right now, there are roughly two ways to set up the slave
machines:
1) Slave machines are virtual machines *on* the test-driver machine.
2)
Hi Geoffrey,
Thank you for your detailed explaining. They are really helpful.
I am thinking of going after the second way, since I have bare metal access
to all the nodes in the cluster, it's probably better to run real slave
machines instead of virtual machines. (correct me if I am wrong)
Each
Yes, A list of Kafka Server host/port pairs to use for establishing the
initial connection to the Kafka cluster
https://kafka.apache.org/documentation.html#newproducerconfigs
On Tue, Jul 14, 2015 at 7:29 PM, Yuheng Du yuheng.du.h...@gmail.com wrote:
Does anyone know what is bootstrap.servers=
Thanks. If I set the acks=1 in the producer config options in
bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance
test7 5000 100 -1 acks=1 bootstrap.servers=
esv4-hcl198.grid.linkedin.com:9092 buffer.memory=67108864 batch.size=8196?
Does that mean for each message
Also, I guess setting the target throughput to -1 means let it be as high
as possible?
On Tue, Jul 14, 2015 at 10:36 AM, Yuheng Du yuheng.du.h...@gmail.com
wrote:
Thanks. If I set the acks=1 in the producer config options in
bin/kafka-run-class.sh
Does anyone know what is bootstrap.servers=
esv4-hcl198.grid.linkedin.com:9092 means in the following test command:
bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance
test7 5000 100 -1 acks=1 bootstrap.servers=
esv4-hcl198.grid.linkedin.com:9092 buffer.memory=67108864
@Jiefu, yes! The patch is functional, I think it's just waiting on a bit of
final review after the last round of changes. You can definitely use it for
your own benchmarking, and we'd love to see patches for any additional
tests we missed in the first pass!
-Ewen
On Tue, Jul 14, 2015 at 10:53
Yuheng,
I would recommend looking here:
http://kafka.apache.org/documentation.html#brokerconfigs and scrolling down
to get a better understanding of the default settings and what they mean --
it'll tell you what different options for acks does.
Ewen,
Thank you immensely for your thoughts, they
I implemented (nearly) the same basic set of tests in the system test
framework we started at Confluent and that is going to move into Kafka --
see the wip patch for KIP-25 here: https://github.com/apache/kafka/pull/70
In particular, that test is implemented in benchmark_test.py:
Hi all,
I was wondering if any of you guys have done benchmarks on Kafka
performance before, and if they or their details (# nodes in cluster, #
records / size(s) of messages, etc.) could be shared.
For comparison purposes, I am trying to benchmark Kafka against some
similar services such as
13 matches
Mail list logo
