Test kit for java
-----------------
Key: QPID-1161
URL: https://issues.apache.org/jira/browse/QPID-1161
Project: Qpid
Issue Type: Bug
Components: Java Tests
Affects Versions: M3
Reporter: Rajith Attapattu
Assignee: Rajith Attapattu
Fix For: M3
Attachments: testkit.patch
The attached Test kit for the java client consists of 2 components.
1) A Simple Perf Test that can be used to,
a) Run a predefined perf report consisting of 8 use cases (see below)
b) Run a producer and a consumer with a number of different options
2) Soak tests that can be run for longer durations (hours or days).
I am planning to add some stress tests to this module as well.
Please note this is not a replacement for the existing perf/systests etc.
But rather a small test kit thats focused on providing a packaged set of tests
that can be quickly deployed on an environment to do quick smoke testing or
easily setup a soak test.
1.0 Perf Kit
------------
1.1 The perf kit can be packaged as an RPM or a tar file and deploy on a target
environment and run the perf report.
Or else a perf report can be automated to run every day or so an record numbers
to catch perf regressions.
1.2 It calculates the following results in msg/sec.
System throuhgput : no_of_msgs / (time_last_msg_rcvd - time_first_msg_send)
Producer rate : no_of_msgs / (time_after_sending - time_before_sending)
Producer rate : no_of_msgs / (time_last_msg_rcvd - time_first_msg_rcvd)
Latency : time_msg_rcvd - time_msg_sent
The test will print min, max and avg latency.
1.3 The test assume that both producer and consumer are run on the same machine
or different machines that are time synced.
1.4 You can also use run_sub.sh and run_pub.sh to run different use cases with
several options.
Please look at TestParams.java for all the configurable options.
1.5 You can also use the test kit to benchmark against any vendor.
2.0 Soak tests
--------------
2.0 This includes a set of soak tests that can be run for a longer duration.
2.1 A typical test will send x-1 messages and the xth message will contain an
"End" marker.
The producer will print the message_id and timestamp for the xth message.
The consumer will reply with an empty message to the replyTo destination
given in the xth message.
The consumer also prints the message_id and timestamp for the xth message.
A typical value for x is 100k
2.2 The feedback loop prevents the producer from overrunning the consumer.
And the printout for every xth message will let you know how many
iterations been completed at any given time.
2.2 The following results can be calculated for these tests.
Memory, CPU for each producer/consumer - scripts will be checked in
Using the message_id and timestamp you can get latency samples for the test
run.
Using the timestamps you can calculate the system throughput rates.
2.2 If you use different machines for producer and consumer the machines have
to be time synced if you want latency samples.
3.0 Perf Test report use cases
-------------------------------
3.1 Please check testkit/bin/perf_report.sh for more details
3.2 A typical test run will send 1000 msgs during warmup and 200k msgs for
result calculation.
Test 1 Trans Queue
Test 2 Dura Queue
Test 3 Dura Queue Sync
Test 4 Topic
Test 5 Durable Topic
Test 6 Fanout
Test 7 Small TX (about 2 msgs per tx)
Test 8 Large TX (about 1000 msgs per tx)
4.0 Soak tests use cases
-------------------------
4.1 Following are the current tests available in the test kit.
4.2 Please refer to the source to see the javadoc and options
1. SimpleProducer/Consumer sends X messages at a time and will wait for
confirmation from producer before proceeding with the next iteration. A no of
options can be configured.
2. MultiThreadedProducer/Consumer does the same thing as above but runs each
session in a separate thread.
It can also send messages transactionally. Again a no of options can be
configured.
3. ResourceLeakTest will setup consumer/producers sends x messages and then
teard down everything and continue again.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
