[
https://issues.apache.org/jira/browse/SOLR-18126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18063226#comment-18063226
]
David Smiley commented on SOLR-18126:
-------------------------------------
Update: I've spent some time with Claude on this for a couple weeks refining
it. I have branch with something working well, and that which fixes a couple
bugs in the benchmarks or the framework for the benchmarks. I think we should
have a CI run that runs the benchmarks to ensure they don't fail. I have an
optimized CLI command for that. I also think such a run should use the
ObjectReleaseTracker -- which I did, which aided fixing a benchmark bug.
I also added an "EmbeddedSolrServer" option since most of our tests *should* be
lower level tests. So that means 3 backends. I then realized that the
"SolrBenchBackend" I created, with 3 implementations, was looking pretty
similar to the SolrClientTestRule abstraction / concept, and that both sides
had implementations that the other side would like (e.g. there's no
RemoteSolrClientTestRule, and there's no JettySolrRunnerBenchBackend). I think
it makes sense to generalize further to simply "SolrBackend" (not necessarily
for benchmarks), and have it be the backend for SolrBenchState (for benchmarks)
and SolrClientTestRule (for tests). This will simplify SolrClientTestRule as
it limits its concerns, and it means investments in backends motivated by
either testing or benchmarking cross-benefit each other. Like, obviously we'd
also like a Docker one to exist. Furthermore, this approach decouples from
JUnit 4, making it easy for a JUnit 5+ user to write an equivalent "rule" based
on a SolrBackend.
> Benchmarks: decouple from MiniSolrCloudCluster; support remote
> --------------------------------------------------------------
>
> Key: SOLR-18126
> URL: https://issues.apache.org/jira/browse/SOLR-18126
> Project: Solr
> Issue Type: Test
> Components: benchmarks
> Reporter: David Smiley
> Assignee: David Smiley
> Priority: Major
>
> Currently all JMH benchmarks in solr/benchmark are hard-wired to
> MiniSolrCloudCluster via MiniClusterState.MiniClusterBenchState. This makes
> it impossible to run benchmarks against a real Solr instance without code
> changes.
> Introduce a SolrBenchBackend interface with MiniClusterBackend (extracts
> existing mini-cluster logic) and RemoteSolrBackend (connects to an existing
> Solr via solr.bench.url system property) implementations. A new
> SolrBenchState JMH @State class replaces MiniClusterState, selecting the
> backend at runtime via a solr.bench.backend system property (minicluster by
> default).
> All existing benchmark classes (SimpleSearch, FilterCache, JsonFaceting,
> NumericSearch, QueryResponseWriters, ExitableDirectoryReaderSearch,
> StreamingSearch, CloudIndexing) are updated to use SolrBenchState. System
> properties previously set on benchmark setup that configure Solr internals
> (e.g. cache settings, merge policy) are converted to collection-level
> creation properties passed through createCollection, keeping them compatible
> with remote backends.
> MiniClusterState.java is deleted as part of this change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
