Hello,
I need to check case with kafka cluster expansion at the integration test.
I see the EmbeddedKafkaCluster and IntegrationTestHarness are popular to use at
he integration tests.
But these test classes don't support test cluster expansion.
What do you suggest to use in this case?
Does
Hi Kafka Experts
I recently worked on kafka perf test, my kafka active/standby clusters and
mirrormake2 are all built with sasl-ssl, kafka-producer-perf-test.sh setup with
sasl-ssl too. I use kafka default settings, say, message.max.size, batch
size...no changes...
Test result shows active
022 at 13:13, Ivanov, Evgeny
wrote:
> Hi everyone,
>
> I ran several performance test below and the overall throughput looks
> good, but I don't understand why it shows huge latency (about 2 seconds).
> Could you please explain what that latency means ?
> It can't be the latency be
Hi everyone,
I ran several performance test below and the overall throughput looks good, but
I don't understand why it shows huge latency (about 2 seconds).
Could you please explain what that latency means ?
It can't be the latency between records or batches. What it is then ?
kafka-producer
ow how
can I delete data from Kafka testcontainers after each test so that I don't
have to destroy and recreate the kafka testcontainer every time.
- Test Container Version - 1.6.2
- Docker Kafka Image Name - confluentinc/cp-kafka:5.2.1
at 9:34 PM mohammad shadab <
mohammad.s.sha...@oracle.com> wrote:
> Hi Luke,
>
> Thanks a lot for your time and assistance.
>
> Some how its wasn't attached, attaching again Kafka 3.0 unit test report.
>
> For my work i need to generate binaries from kafka source code whic
Hi Luke,
Thanks a lot for your time and assistance.
Some how its wasn't attached, attaching again Kafka 3.0 unit test report.
For my work i need to generate binaries from kafka source code which i am doing
locally , so there is no PR in repo.
I am just running unit test( i think that should
from git as i tool from
> https://kafka.apache.org/downloads
> <https://urldefense.com/v3/__https://kafka.apache.org/downloads__;!!ACWV5N9M2RV99hQ!e8QWe1ktCCKG2r0b2uBU9ydXmVOXfq-7ERFXY0M6KKBWBvdg5Az6ZPZjUZ47_TgqfcjQbw$>
> .
>
> attaching kafka 3.0 unit test report.
>
>
&
9hQ!e8QWe1ktCCKG2r0b2uBU9ydXmVOXfq-7ERFXY0M6KKBWBvdg5Az6ZPZjUZ47_TgqfcjQbw$>
.
attaching kafka 3.0 unit test report.
* Gradle 7.3.3
*
* java version "11.0.13" 2021-10-19 LTS
* Java(TM) SE Runtime Environment 18.9 (build 11.0.13+10-LTS-370)
* Java HotSpot(TM) 64-Bit Server VM 18.9 (buil
not sure if it works well in v2.8 (because of different
dependencies)
Thank you.
Luke
On Fri, Dec 24, 2021 at 5:44 PM mohammad shadab <
mohammad.s.sha...@oracle.com> wrote:
> Hi all,
>
> Need some assistance as unit test is falling for kafka 2.8.0 , below are
> detail. Appreciate any
Hi all,
Need some assistance as unit test is falling for kafka 2.8.0 , below are
detail. Appreciate any guidance , Please redirect me to correct audience if i
am firing this email to wrong audience.
I have downloaded kafka 2.8.0 source code (from
https://kafka.apache.org/downloads
: Saturday, February 6, 2021 at 8:22 PM
To: users@kafka.apache.org
Subject: Re: Kafka 2.7.0 processor API and streams-test-utils changes
Hello Upesh,
I’m sorry for the trouble. This was my feature, and my oversight. I will update
the docs on Monday.
The quick answer is that there is also a new
with the state stores:
https://kafka.apache.org/27/javadoc//org/apache/kafka/streams/processor/api/MockProcessorContext.html#getStateStoreContext--
Here is an example of how to use it:
https://github.com/apache/kafka/blob/trunk/streams/test-utils/src/test/java/org/apache/kafka/streams/test/wordcount
>
> https://kafka.apache.org/27/documentation/streams/developer-guide/testing.html#unit-testing-processors
>
>
>
> However, it seems that the MockProcessorContext and possibly other test
> suite classes have not been updated for the new API changes, such as the
>
/testing.html#unit-testing-processors
However, it seems that the MockProcessorContext and possibly other test suite
classes have not been updated for the new API changes, such as the following
methods:
store.init(context, store);
context.register(store, null);
Can someone point me in the right
i how do use java kafka-client can test acks 0 and 1 and all between
difference???
]
From: Sunil CHAUDHARI
Sent: Tuesday, March 3, 2020 2:43 PM
To: users@kafka.apache.org
Subject: Producer-Perf-test
Hi,
I have done performance testing of Kafka cluster using
kafka-producer-perf-test.sh
I created diff type of topics and did perf testing. Example: MB1P1R= MB is my
topic name
It's okay. You could probably take the screenshot and share it.
On Tue, 3 Mar 2020, 20:13 sunil chaudhari,
wrote:
> Hi Himanshu,
> Sorry but I pasted from excel. Dont know how it got messed up?
>
> Will resend it.
>
> On Tue, 3 Mar 2020 at 6:48 PM, Himanshu Shukla <
>
Hi Himanshu,
Sorry but I pasted from excel. Dont know how it got messed up?
Will resend it.
On Tue, 3 Mar 2020 at 6:48 PM, Himanshu Shukla
wrote:
> could you please share the result in some proper way? Each field is line by
> line.
>
> On Tue, Mar 3, 2020 at 2:43 PM Sunil CHAUDHARI
> wrote:
>
could you please share the result in some proper way? Each field is line by
line.
On Tue, Mar 3, 2020 at 2:43 PM Sunil CHAUDHARI
wrote:
> Hi,
> I have done performance testing of Kafka cluster using
> kafka-producer-perf-test.sh
> I created diff type of topics and did perf testing. Example:
Hi,
I have done performance testing of Kafka cluster using
kafka-producer-perf-test.sh
I created diff type of topics and did perf testing. Example: MB1P1R= MB is my
topic name with 1 Partition and 1 replica.
I have 3 nodes cluster.
My Observations:
* When I increased partitions, and keep
Hi,
I have run consumer performance test on my kafka cluster.
Can you please help me to understand below parameters? Basically I don't know
whats unit of those parameters? I cant assume it blindly
Its only given in 2 columnts, the "Metric Name" and its "Valu
Hi,
When I run Consumer-perf-test from kafka bin directory to measure
performance, thne in the output file, it shows only headers.
Ot doesnt prints their values.
Any thought on this?
When I run producer-perf-test then it prints detailed information like,
Number of records read:
Number of records
spectives>
On Thu, Feb 20, 2020 at 3:39 PM Jp Silva wrote:
> Hi,
>
> I'm using kafka-consumer-perf-test but I'm getting an error if I add the
> --print-metrics option.
>
> Here's a snippet of my output including the error:
>
> consumer-fetch-manager-metrics:fetch-size-m
Hi,
I'm using kafka-consumer-perf-test but I'm getting an error if I add the
--print-metrics option.
Here's a snippet of my output including the error:
consumer-fetch-manager-metrics:fetch-size-max:{client-id=consumer-perf-consumer-99250-1
/ test for lost
messages?
kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group KafkaMirror
--zkconnect source-cluster-zookeeper:2181 --topic test-topic
I can understand that that is the only way we can check the mirroring?
Best Regards,
--
Kamen Tarlov
/ test for lost
messages?
kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group KafkaMirror
--zkconnect source-cluster-zookeeper:2181 --topic test-topic
I can understand that that is the only way we can check the mirroring?
Best Regards,
--
Kamen Tarlov
ausing this issue?
>
> Sincerely,
> Anindya Haldar
> Oracle Responsys
>
>
>> On Oct 28, 2019, at 11:52 AM, M. Manna wrote:
>>
>> Hi,
>>
>> Per test is based on a set of tuning parameters e.g. batch.size, axes,
>> partitions, network latency etc
’, and
‘transaction-duration-ms’. Both are set to 1 hour, and the failure we see
happens within a few minutes. Is there a third timeout value that is expiring
and causing this issue?
Sincerely,
Anindya Haldar
Oracle Responsys
> On Oct 28, 2019, at 11:52 AM, M. Manna wrote:
>
> Hi,
>
> Pe
Hi,
Per test is based on a set of tuning parameters e.g. batch.size, axes,
partitions, network latency etc. Your transactions are failing because your
batch has expired, (or at least, that’s what it shows on the log). You have
to tune your request timeout and batch.size correctly to improve
Anyone with a pointer on this? Do transactions work reliably with Kafka perf
test tools? If yes, then is there a way to make it work in this scenario?
Sincerely,
Anindya Haldar
Oracle Responsys
> On Oct 25, 2019, at 2:51 PM, Anindya Haldar wrote:
>
> We are evaluating Kafka for so
We are evaluating Kafka for some of our use cases. As part of that effort I am
trying to run an experiment with a cluster we have set up, and using the
producer perf test tool supplied with the binaries.
Here’s the cluster info:
Runs in Kubernetes, with 4 CPUs, 32 GB RAM, 100 GB log space
Hello,
We are trying to test our GCP 3 node cluster based on some testing done
previously here
https://cwiki.apache.org/confluence/display/KAFKA/Performance+testing
We have seen some inconsistencies in results. With 2.3.0 we are running
with the following producer params(single-threaded
I am new to using Kafka and wondering if any of experts out there can let me
know how i can run a standalone Network Unit test for Kafka Server. I see this
class SocketServerTest in Unit tests for Networking but not able to figure out
the commands to use it or i need to write any test program
Huh! Never realised our ASF has bots working behind the scene. Thanks
Matthias :)
On Wed, 22 May 2019 at 12:27, Matthias J. Sax wrote:
> Just leave a commend
>
> Retest this please.
>
> on Github and it will trigger a retest. No special permission required.
>
>
> -Matthias
>
>
> On 5/22/19
Just leave a commend
Retest this please.
on Github and it will trigger a retest. No special permission required.
-Matthias
On 5/22/19 1:10 PM, M. Manna wrote:
> Hello,
>
> One of the integration tests failed due to a timeout issue.
>
> https://github.com/apache/kafka/pull/6771
>
>
Hello,
One of the integration tests failed due to a timeout issue.
https://github.com/apache/kafka/pull/6771
Could someone with access please re-trigger the build ? This didn't fail
for me when I checked previously.
Thanks,
and balance them ahead of time.
-hans
On Wed, May 15, 2019 at 8:45 AM M. Manna wrote:
> Hello,
>
> I am trying to do some performance testing using Kafka-Consumer-Perf-Test.
> Could somone please help me understand whether my setup is correct?
>
> 1) I would like to ru
Hello,
I am trying to do some performance testing using Kafka-Consumer-Perf-Test.
Could somone please help me understand whether my setup is correct?
1) I would like to run a benchmark test to have 10 publishers publishing
100 messages (4MB) each and 10 subscribers.
2) For the above, do I need
In Kafka 2.1, the `Serializer` and `Deserializer` interface were
changed:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=87298242
It seems, that updating to 2.2.0 for the test driver does not update
your dependency to `kafka-client` package. Try to add this dependency
explicitly
It seems like the Kafka test driver is not compatible with the latest
kafka-streams API. When I use the following two dependencies together.
org.apache.kafka
kafka-streams-test-utils
2.2.0
test
org.apache.kafka
kafka
Hi ,
I would like to add a integration test for kafka rebalance scenario and
make sure retries are working as expected if enough replicas are not there
in my kafka streams application .
Do you have any info on how can i achieve this .
I was checking TestUtils class in org.apache.kafka.test
, 2018 at 1:19 AM Localhost shell <
universal.localh...@gmail.com> wrote:
> Hello All,
>
> I am trying to perform a benchmark test in our kafka env. I have played
> with few configurations such as request.timeout.ms and max.block.ms and
> throughout but not ab
Hello All,
I am trying to perform a benchmark test in our kafka env. I have played
with few configurations such as request.timeout.ms and max.block.ms and
throughout but not able to avoid the error:
org.apache.kafka.common.errors.TimeoutException: The request timed out
Sweet! I think this pretty much wraps up all the discussion points.
I'll update the KIP with all the relevant aspects we discussed and call for
a vote.
I'll also comment on the TopologyTestDriver ticket noting this modular test
strategy.
Thanks, everyone.
-John
On Fri, Mar 9, 2018 at 10:57 AM
Hi,
I need to write unit tests with Compacted Topics in local cluster. Has
anyone done something like that? Any tips/guidance will be much appreciated.
Thanks
Sirisha
, that is the
responsibility of the Kafka Streams developers and contributors.
However, for the developers and contributors of Kafka Streams, the need to
test the internals of how Streams works is the primary concern and could at
times require different logic or available methods from a given mock
I think what you're suggesting is to:
1. compile the main streams code, but not the tests
2. compile test-utils (and compile and run the test-utils tests)
3. compile and run the streams tests
This works in theory, since the test-utils depends on the main streams
code, but not the streams tests
MockProcessorContext is only used in unit tests, and hence we should be
able to declare it as a test dependency of `streams` in gradle build file,
which is OK.
Guozhang
On Thu, Mar 8, 2018 at 3:32 PM, John Roesler <j...@confluent.io> wrote:
> Actually, replacing the MockProcess
On Wed, Mar 7, 2018 at 8:04 PM, John Roesler <j...@confluent.io> wrote:
> >>
> >>> On Wed, Mar 7, 2018 at 8:03 PM, John Roesler <j...@confluent.io>
> wrote:
> >>>
> >>>> Thanks Ted,
> >>>>
> >>>&
Isn't MockProcessorContext in o.a.k.test part of the unit-test package
but not the main package?
This should resolve the dependency issue.
-Matthias
On 3/8/18 3:32 PM, John Roesler wrote:
> Actually, replacing the MockProcessorContext in o.a.k.test could be a bit
> tricky, since it woul
M, John Roesler <j...@confluent.io> wrote:
>>>
>>>> Thanks Ted,
>>>>
>>>> Sure thing; I updated the example code in the KIP with a little
>> snippet.
>>>>
>>>> -John
>>>>
>>>> On Wed, Mar 7, 2018
hird module that depends on both
streams and test-utils. Yuck!
Thanks,
-John
On Thu, Mar 8, 2018 at 3:16 PM, John Roesler <j...@confluent.io> wrote:
> Thanks for the review, Guozhang,
>
> In response:
> 1. I missed that! I'll look into it and update the KIP.
>
> 2.
M, Ted Yu <yuzhih...@gmail.com> wrote:
> > >
> > >> Looks good.
> > >>
> > >> See if you can add punctuator into the sample code.
> > >>
> > >> On Wed, Mar 7, 2018 at 7:10 PM, John Roesler <j...@confluent.io>
> wrote:
good.
>>
>> See if you can add punctuator into the sample code.
>>
>> On Wed, Mar 7, 2018 at 7:10 PM, John Roesler <j...@confluent.io> wrote:
>>
>> > Dear Kafka community,
>> >
>> > I am proposing KIP-267 to augment the public Streams
0 PM, John Roesler <j...@confluent.io> wrote:
>
> > Dear Kafka community,
> >
> > I am proposing KIP-267 to augment the public Streams test utils API.
> > The goal is to simplify testing of Kafka Streams applications.
> >
> > Please find details in the
> > wi
Looks good.
See if you can add punctuator into the sample code.
On Wed, Mar 7, 2018 at 7:10 PM, John Roesler <j...@confluent.io> wrote:
> Dear Kafka community,
>
> I am proposing KIP-267 to augment the public Streams test utils API.
> The goal is to simplify testin
Dear Kafka community,
I am proposing KIP-267 to augment the public Streams test utils API.
The goal is to simplify testing of Kafka Streams applications.
Please find details in the
wiki:https://cwiki.apache.org/confluence/display/KAFKA/KIP-267%3A+Add+Processor+Unit+Test+Support+to+Kafka+Streams
Also using these dependencies
- Gradle: org.springframework.kafka:spring-kafka-test:1.1.7.RELEASE
- Gradle: org.springframework.kafka:spring-kafka:1.3.2.RELEASE
On Wed, Feb 14, 2018 at 2:13 PM, Ian Ewing <ian.ew...@gmail.com> wrote:
> From my build.gradle:
>
&
ngframework.boot:spring-boot-starter-test")
compile("org.springframework.kafka:spring-kafka:1.3.2.RELEASE")
testCompile("org.springframework.kafka:spring-kafka-test")
}
And this is from my project structure. I wonder if that is part of the
problem, having .10 and .11?
LoginType was in 0.10.x release.
This seems to indicate Kafka version mismatch.
Can you check the dependencies of your test ?
Thanks
On Tue, Feb 13, 2018 at 8:03 PM, Ian Ewing <ian.ew...@gmail.com> wrote:
> I have been trying to figure out how to unit test a kafka producer. Shou
I have been trying to figure out how to unit test a kafka producer. Should
take in a simple integer and perform some addition. Followed what I could
find on spring kafka unit testing but keep running into this error:
19:53:12.788 [main] ERROR kafka.server.KafkaServer - [Kafka Server 0],
Fatal
Topic: repl3part5Partition: 4Leader: 3Replicas: 3,2,1
>>> Isr: 3,2,1
>>>
>>> I have a producer for this topic:
>>>
>>> bin/kafka-console-producer.sh --broker-list 192.168.112.31:9092,
>>> 192.168.112.32:9092,192.168.112.33:9092 --top
192.168.112.31:9092,192.168.112.32:9092,192.168.112.33:9092 --topic
repl3part5 --consumer-property group.id=zoran_1
Every message that is sent by producer gets collected by consumer. So
far - so good.
Now I would like to test fail over of the kafka servers. If I put
down blade 3 kafka service, I
This is not my question, but I saw it on Stack Overflow yesterday and have
been wondering about it: https://stackoverflow.com/questions/48523972/group-
consumer-cannot-consume-messages-if-kafka-service-on-specific-node-in-test.
Anyone else seen behavior like this?
--topic
repl3part5 --consumer-property group.id=zoran_1
Every message that is sent by producer gets collected by consumer. So
far - so good.
Now I would like to test fail over of the kafka servers. If I put down
blade 3 kafka service, I get consumer warnings but all produced
messages are still
.
Now I would like to test fail over of the kafka servers. If I put down
blade 3 kafka service, I get consumer warnings but all produced messages
are still consumed.
[2018-01-30 14:30:01,203] WARN [Consumer clientId=consumer-1,
groupId=zoran_1] Connection to node 3 could not be established. Broker
for the discussion and voting!
-Matthias
On 1/18/18 4:17 PM, Matthias J. Sax wrote:
> I added the new method to the KIP and also updated the PR.
>
> -Matthias
>
> On 1/18/18 10:48 AM, Guozhang Wang wrote:
>> @Matthias
>>
>> This comes to me while reviewing another using
I added the new method to the KIP and also updated the PR.
-Matthias
On 1/18/18 10:48 AM, Guozhang Wang wrote:
> @Matthias
>
> This comes to me while reviewing another using the test driver: could we
> add a `Map<String, StateStore> allStateStores()` to the
> `TopologyTe
+1 (non-binding)
-James
Sent from my iPhone
> On Jan 17, 2018, at 6:09 PM, Matthias J. Sax <matth...@confluent.io> wrote:
>
> Hi,
>
> I would like to start the vote for KIP-247:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-247%3A+Add+public+test
start the vote for KIP-247:
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 247%3A+Add+public+test+utils+for+Kafka+Streams
> >
> >
> > -Matthias
> >
> >
>
Thanks for the KIP.
+1
-Bill
On Wed, Jan 17, 2018 at 9:09 PM, Matthias J. Sax <matth...@confluent.io>
wrote:
> Hi,
>
> I would like to start the vote for KIP-247:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 247%3A+Add+public+test+utils+for+Kafka+Streams
>
>
> -Matthias
>
>
+1
Original message From: Guozhang Wang <wangg...@gmail.com>
Date: 1/17/18 10:38 PM (GMT-08:00) To: users@kafka.apache.org Subject: Re:
[VOTE] KIP-247: Add public test utils for Kafka Streams
+1 (binding).
On Wed, Jan 17, 2018 at 6:09 PM, Matthias J. Sax
+1 (binding).
On Wed, Jan 17, 2018 at 6:09 PM, Matthias J. Sax <matth...@confluent.io>
wrote:
> Hi,
>
> I would like to start the vote for KIP-247:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 247%3A+Add+public+test+utils+for+Kafka+Streams
>
>
> -Matthias
>
>
--
-- Guozhang
Hi,
I would like to start the vote for KIP-247:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-247%3A+Add+public+test+utils+for+Kafka+Streams
-Matthias
signature.asc
Description: OpenPGP digital signature
. However,
> there are no plans atm to integrate the test driver with a schema
> registry. Note, that Apache Kafka does not provide a schema registry by
> itself. If we integrate one RS, we need to integrate all of them -- that
> is something Apache Kafka should not do from my point of view.
>
&g
Saïd,
sorry for late reply. Avro records are basically supported. However,
there are no plans atm to integrate the test driver with a schema
registry. Note, that Apache Kafka does not provide a schema registry by
itself. If we integrate one RS, we need to integrate all of them
ocessing of single-partitioned input topics purely in-memory (the
> driver is basically a mock for a StreamThread). This is sufficient to
> test basic business logic. For more complex topologies that are actually
> divided into sub-topologies and connected via topics, the driver detects
> t
Hello,
We are using kafka for pub sub and want test performance of entire system.
Is there any tool readily available in kafka world which can simulate
multiple publishers and subscribers tp measure latency and throughput
considering custom application layer?
Any guidelines around this would
Colin,
the TopologyTestDriver does not connect to any broker and simulates
processing of single-partitioned input topics purely in-memory (the
driver is basically a mock for a StreamThread). This is sufficient to
test basic business logic. For more complex topologies that are actually
divided
Forgot dev-list...
Forwarded Message
Subject: Re: [DISCUSS] KIP-247: Add public test utils for Kafka Streams
Date: Tue, 16 Jan 2018 13:56:38 -0800
From: Matthias J. Sax <matth...@confluent.io>
Organization: Confluent Inc
To: users@kafka.apache.org
Thanks a lot for the co
Thanks a lot for the comments.
@Guozhang: I updated the KIP accordingly. With regard to potential
client test-utils, I agree, but not sure how to resolve it. I guess, we
just need to move the class for this case later on. (One reason to
annotate all classes with @Evolving)
@Bill: The new
res in the scope of this KIP?
On Thu, Jan 11, 2018 at 9:06 PM, Matthias J. Sax <matth...@confluent.io>
wrote:
> Dear Kafka community,
>
> I want to propose KIP-247 to add public test utils to the Streams API.
> The goal is to simplify testing of Kafka Streams applications.
&g
y
> > client mock is not thread safe and thus in the different processors nodes
> > when deserializing it will not work...
> >
> > In my unit tests I wrapped the mock schema registry client into a
> singleton
> > but this solution is not enough satisfying.
> &
On Thu, Jan 11, 2018, at 18:06, Matthias J. Sax wrote:
> Dear Kafka community,
>
> I want to propose KIP-247 to add public test utils to the Streams API.
> The goal is to simplify testing of Kafka Streams applications.
>
> Please find details in the wiki:
> https://cwiki.
i, Jan 12, 2018 at 3:06 AM Matthias J. Sax <matth...@confluent.io>
> wrote:
>
> > Dear Kafka community,
> >
> > I want to propose KIP-247 to add public test utils to the Streams API.
> > The goal is to simplify testing of Kafka Streams applications.
>
is not enough satisfying.
Thanks in advance, regards :-)
On Fri, Jan 12, 2018 at 3:06 AM Matthias J. Sax <matth...@confluent.io>
wrote:
> Dear Kafka community,
>
> I want to propose KIP-247 to add public test utils to the Streams API.
> The goal is to simplify testin
Dear Kafka community,
I want to propose KIP-247 to add public test utils to the Streams API.
The goal is to simplify testing of Kafka Streams applications.
Please find details in the wiki:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-247%3A+Add+public+test+utils+for+Kafka+Streams
Revin Chalil" <rcha...@expedia.com> wrote:
> >
> > We are testing kafka’s performance with the real prod data and plan
> to
> > test things like the below. We would have producers publishing and
> > consumers processing production data on a separate non
details. Just turn on the console reporter for
the easiest start.
Its not much code to put out a repeatable test with the metrics you care
for. The only other test I'd look to add is scaling producers and
consumers, that's where kafka really shines. :)
Hope that helps some.
On Fri, Sep 15, 2017
@expedia.com> wrote:
>
> We are testing kafka’s performance with the real prod data and plan to
> test things like the below. We would have producers publishing and
> consumers processing production data on a separate non-prod kafka cluster.
>
>
> * Impact of num
We are testing kafka’s performance with the real prod data and plan to test
things like the below. We would have producers publishing and consumers
processing production data on a separate non-prod kafka cluster.
* Impact of number of Partitions per Topic on throughput and latency
kafka is a distributed, partitioned, replicated commit log service message
queue, i am interesting how kafka test.
transaction message ,exactly once message ,replication ,any document about
kafka test?
how to ensure kafka work as except
Yeah, I'll open a PR.
2017-07-12 11:07 GMT-06:00 Matthias J. Sax <matth...@confluent.io>:
> Ups. That is definitely a bug in the test. Thanks for pointing it out!
>
> Do you wanna open a PR it? If not, I can take care, too.
>
> -Matthias
>
> On 7/12/17 8:48 AM, Joel
Ups. That is definitely a bug in the test. Thanks for pointing it out!
Do you wanna open a PR it? If not, I can take care, too.
-Matthias
On 7/12/17 8:48 AM, Joel Dice . wrote:
> After some debugging, I figured it out. The name of my custom store
> was "mapStore", so the st
After some debugging, I figured it out. The name of my custom store
was "mapStore", so the store tried to log changes to a topic with that
name, but my test case never created such a topic. Unsurprisingly,
the producer couldn't get metadata for a nonexistent topic, so it
failed with
Seems Streams cannot connect (or looses connection) to the brokers. Not
sure why.
You can also have a look here for our own EOS Streams integration test:
https://github.com/apache/kafka/blob/trunk/streams/src/test/java/org/apache/kafka/streams/integration/EosIntegrationTest.java
-Matthias
Hi all,
I wrote an integration test to see how Kafka Streams handles exactly
once processing with a custom state store, and it's failing with this
error:
org.apache.kafka.common.errors.TimeoutException: Failed to update
metadata after 6 ms
I'm not sure whether this is a bug in my code
Hi,
I ran a JUnit test where i delivered a payload of ~5MB file to a Kafka
cluser (3 broker, 3 zookeeper). These are all setup on my laptop. My test
config for producer is the following:
max.request.size=5242880
> batch.size=8192
> key.seri
test
1 - 100 of 282 matches
Mail list logo
