Guys, I think you're missing the point. Of course you can transfer small payloads with Solarflare/OpenOnload or IB/VMA in single digit microseconds in simple ping-pong tests. That is standard stuff in the industry in which I work.
This is NOT the kind of testing I'm doing. This Message Bus Load Testing involves simulating multiple clients serializing and sending medium to large messages from a Publisher Host (Host A) at predefined throughput levels to one or more Topics on a Message Broker (Host B), which then may itself perform more processing on the message(s) for replication or persistence purposes, and then sink to interested Subscribers on a separate host. This is far different than the kind of ping-pong tests you guys appear to be describing. This involves much more than kernel bypass, PCIE, NIC HW, serialization, and propagation delay from one hop to the next. On Sat, Dec 21, 2019, 4:36 PM Wojciech Kudla <[email protected]> wrote: > > However, the messaging platform deals with transferring pub/sub > messages that can vary from 1KB to 1MB in size – that is decidedly **not** > in the microsecond realm, even if it were on an Infiniband network > communicating via ibverbs > > I'm afraid I have to disagree with this statement. > Contemporary solutions (eg SolarFlare) are capable of exactly that. What's > more - you can see sub-microsecond latency for small payloads (in synthetic > benchmarks). > Single digit microsecond latency is actually expected these days for > disseminating (market) data over a single hop in local network. > > > On Sat, 21 Dec 2019, 20:10 Mark Dawson, <[email protected]> wrote: > >> Peter, >> >> >> >> Yes, I do work in the HFT, and we do use low-latency infrastructure for >> exchange-facing infrastructure. However, the messaging platform deals with >> transferring pub/sub messages that can vary from 1KB to 1MB in size – that >> is decidedly **not** in the microsecond realm, even if it were on an >> Infiniband network communicating via ibverbs. >> >> >> >> My interest is in response time measurements of various Message Bus >> products, so we’re talking millisecond scale for which tools like JMeter >> are more than enough. My issue is with the error-prone reporting with their >> Response Time Percentile Graph that I’ve noticed so far. >> >> >> >> I think Gil’s suggestion gets me closer to what I thought I wanted. But >> the fact that I can’t make use of exponentially distributed request >> intervals, like what the new Precise Throughput Timer permits, makes me >> think we’ll have to craft our own (a fraught exercise all its own). >> >> >> >> >> >> *From: *Peter Booth <[email protected]> >> *Sent: *Saturday, December 21, 2019 1:21 PM >> *To: *mechanical-sympathy <[email protected]> >> *Subject: *Re: JMeter and HdrHistogram Integration >> >> >> >> Mark, >> >> >> >> I don't know anything about your use-case but the fact that you're >> posting on mechanical sympathy >> >> suggests that latency is important to you. If you are talking about low >> latency messaging, such as that >> >> found n electronic trading then you should forget JMeter. Depending upon >> the age of your infrastructure, >> >> the use of lower latency NICs like Solarflare, Mellanox, low latency >> switches, TCP offload >> >> you could be interested in latency measurements in the low numbers of >> microseconds. >> >> >> >> This testing is harder than it sounds when you consider the impact >> messages of varying sizes, realistic >> >> topologies, slow consumers, different reliability constraints, ... >> >> It's easy to do this badly ( I have many times), and hard to do well. >> >> >> >> I don't know your business context. If it's electronic trading and you >> are looking at >> >> low latency messaging products *(like 60East's AMPS, 29West lbm (now >> Informatica), Tibco's FTL,* >> >> *IBM's low latency messaging (now Cofinity), Aeron, ZeroMQ, Solace) *then >> this is a solved >> >> problem. All the above come with benchmarking tools, but the best >> designed messaging benchmarks >> >> that I have seen are those done by STAC Labs. >> >> >> >> The best independent benchmarking that I have seen is that performed by >> STAC Labs. >> >> They have benchmarked many of the products I listed. See >> https://stacresearch.com/stac-testing-tools >> >> >> >> Hope this helps. >> >> >> >> Feel free to email me directly if you want to chat. >> >> >> >> Peter >> >> peter_booth *at* me.com >> >> >> On Friday, December 20, 2019 at 4:18:12 PM UTC-5, Mark E. Dawson, Jr. >> wrote: >> >> So our company is evaluating a set of messaging platforms, and we're in >> the process of defining non-functional requirements. In preparation for >> evaluating performance, I was considering suggesting JMeter since it >> appears to support testing messaging platforms (with several specific >> tutorials online). However, these tutorials show the Response Time by >> Percentile graphs from the tool, and they all appear to show evidence of >> CO. >> >> Does anyone know if the latest versions include support for HdrHistogram >> either out-of-the-box or via extra configuration? >> >> -- >> You received this message because you are subscribed to the Google Groups >> "mechanical-sympathy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web, visit >> https://groups.google.com/d/msgid/mechanical-sympathy/c84555b9-5ed1-42a9-a844-fe1204cdf16a%40googlegroups.com >> <https://groups.google.com/d/msgid/mechanical-sympathy/c84555b9-5ed1-42a9-a844-fe1204cdf16a%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "mechanical-sympathy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web, visit >> https://groups.google.com/d/msgid/mechanical-sympathy/5dfe7c18.1c69fb81.9b697.e43e%40mx.google.com >> <https://groups.google.com/d/msgid/mechanical-sympathy/5dfe7c18.1c69fb81.9b697.e43e%40mx.google.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "mechanical-sympathy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web, visit > https://groups.google.com/d/msgid/mechanical-sympathy/CAHNMKAqd1X3SB2Cir7oOmwkOjaZ%2Bu73gRUes-h1hcKDCTxyKVw%40mail.gmail.com > <https://groups.google.com/d/msgid/mechanical-sympathy/CAHNMKAqd1X3SB2Cir7oOmwkOjaZ%2Bu73gRUes-h1hcKDCTxyKVw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web, visit https://groups.google.com/d/msgid/mechanical-sympathy/CAFvqqVd4-Dw_Tq7bSiqLc4vf5GpA15v4gZtdqAJqcviAtdj%2Bzg%40mail.gmail.com.
