Two solutions to wrk2 being a http only tool: 1. wrk2 is open source and a very nicely laid out source structure. You can modify the source and replace the http bit with driver for your messaging platform (assuming there is a C binding available).
2. Write a simple web server (using vertx for example) and invoke your messaging subject from there. You will need to take care of instrumenting the server and accumulate the metrics so you can subtract the overhead. This approach also gives you the added benefit of being able to deploy multiple instances of the carefully tuned web server behind an aws elastic load balancer (or a physical one if you are not running this in AWS). This allows you to truly simulate the load without mistakenly reusing resources and connections from a single node running wrk2 with your own messaging driver). You will also need to tune the os. Plus cpu pinning for workload and Ethernet IRQ (interrupt request) handling and a few other things like disk, connection pooling, tcp etc. You are going to have fun either way, I am always excited about work like this :) Sent from my iPhone > On Dec 21, 2019, at 2:12 PM, Mark Dawson <[email protected]> wrote: > > > Faraz, > > I looked at wrk2 last week but it appears to only support HTTP. > > >> On Sat, Dec 21, 2019 at 2:10 PM Faraz Babar <[email protected]> wrote: >> Correctly instrumented performance testing makes test subjects act like >> quantum particles. The act of observation changes the result. If you are >> only interested in black box testing at the boundary, I have had some luck >> with wrk2, but do keep in mind, there are so many different variables from >> message size to network config to proximity between nodes that precise >> accounting of all these variables in black box testing is easy to get wrong. >> And this does not even get into garbage collection or vm overheads if these >> variables are involved. Proceed with caution. >> >> Sent from my iPhone >> >>>> On Dec 21, 2019, at 12:21 PM, Peter Booth <[email protected]> wrote: >>>> >>> >>> Mark, >>> >>> I don't know anything about your use-case but the fact that you're posting >>> on mechanical sympathy >>> suggests that latency is important to you. If you are talking about low >>> latency messaging, such as that >>> found n electronic trading then you should forget JMeter. Depending upon >>> the age of your infrastructure, >>> the use of lower latency NICs like Solarflare, Mellanox, low latency >>> switches, TCP offload >>> you could be interested in latency measurements in the low numbers of >>> microseconds. >>> >>> This testing is harder than it sounds when you consider the impact messages >>> of varying sizes, realistic >>> topologies, slow consumers, different reliability constraints, ... >>> It's easy to do this badly ( I have many times), and hard to do well. >>> >>> I don't know your business context. If it's electronic trading and you are >>> looking at >>> low latency messaging products (like 60East's AMPS, 29West lbm (now >>> Informatica), Tibco's FTL, >>> IBM's low latency messaging (now Cofinity), Aeron, ZeroMQ, Solace) then >>> this is a solved >>> problem. All the above come with benchmarking tools, but the best designed >>> messaging benchmarks >>> that I have seen are those done by STAC Labs. >>> >>> The best independent benchmarking that I have seen is that performed by >>> STAC Labs. >>> They have benchmarked many of the products I listed. See >>> https://stacresearch.com/stac-testing-tools >>> >>> Hope this helps. >>> >>> Feel free to email me directly if you want to chat. >>> >>> Peter >>> peter_booth *at* me.com >>> >>>> On Friday, December 20, 2019 at 4:18:12 PM UTC-5, Mark E. Dawson, Jr. >>>> wrote: >>>> So our company is evaluating a set of messaging platforms, and we're in >>>> the process of defining non-functional requirements. In preparation for >>>> evaluating performance, I was considering suggesting JMeter since it >>>> appears to support testing messaging platforms (with several specific >>>> tutorials online). However, these tutorials show the Response Time by >>>> Percentile graphs from the tool, and they all appear to show evidence of >>>> CO. >>>> Does anyone know if the latest versions include support for HdrHistogram >>>> either out-of-the-box or via extra configuration? >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "mechanical-sympathy" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> To view this discussion on the web, visit >>> https://groups.google.com/d/msgid/mechanical-sympathy/c84555b9-5ed1-42a9-a844-fe1204cdf16a%40googlegroups.com. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "mechanical-sympathy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web, visit >> https://groups.google.com/d/msgid/mechanical-sympathy/9956407B-D81B-4890-A0DA-B0210D997936%40gmail.com. > > -- > You received this message because you are subscribed to the Google Groups > "mechanical-sympathy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web, visit > https://groups.google.com/d/msgid/mechanical-sympathy/CAFvqqVdALxzd6qsw3CVscCo19pBU_CTubX254bVY04qvJGCL5Q%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web, visit https://groups.google.com/d/msgid/mechanical-sympathy/47471061-21F4-4AC0-9B5B-6074D3AC34EA%40gmail.com.
