[
https://issues.apache.org/jira/browse/SOLR-16525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18060713#comment-18060713
]
Jason Gerlowski commented on SOLR-16525:
----------------------------------------
bq. I am looking to improve it instead of get distracted by competing
(non-complementary) efforts. What does Gatling get us that JMH doesn't?
I asked Ishan a very similar question on this same ticket years back; you can
find [his answer a few page-lengths
above|https://issues.apache.org/jira/browse/SOLR-16525?focusedCommentId=17654032&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17654032].
To rephrase and add a bit more detail to his answer...
JMH and Gatling (or "solr-bench" for that matter) are targeting totally
disjoint sets of usecases and questions.
JMH is a microbenchmarking suite. It's great at running small, targeted
benchmarks in a way that lets users gather profiling data like flamegraphs.
You'd use it to find the hotspot within a method invocation, or figure out
where CPU time is being spent. It's the performance testing equivalent of a
unit test.
Gatling, JMeter, "solr-bench", "rally", "luceneutil", etc. are load-testing
frameworks. (Idk if this is a term, but I call them "macrobenchmarks") They're
designed to put large amounts of load on external distributed systems and
measure the response. You'd use them to simulate realistic user-traffic in an
end-to-end way, to measure performance changes over time, or detect
end-user-visible regressions across software versions. They're the performance
testing equivalent of an integration test or smoketest.
That's all to say - Gatling isn't a distraction from JMH or vice versa.
They're complementary pieces of a much larger puzzle.
> Periodic performance benchmarking
> ---------------------------------
>
> Key: SOLR-16525
> URL: https://issues.apache.org/jira/browse/SOLR-16525
> Project: Solr
> Issue Type: Task
> Reporter: Ishan Chattopadhyaya
> Priority: Major
> Attachments: Screenshot from 2023-01-14 17-39-42.png, Screenshot from
> 2023-02-08 04-15-34.png, image-2022-11-09-13-23-21-131.png,
> logs-stress-facets-local.json-b9f5f9ffed8e0c1af679911db6af698d18ab8fe1.zip,
> logs-stress-facets-local.json-ebc823a599160914f7f0f2694f64a53d9cd20268.zip,
> periodic-benchmarks.odp, periodic-benchmarks.pdf
>
>
> This issue is about periodic performance testing of Solr commits against
> pre-written suites of performance tests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
