[
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16020355#comment-16020355
]
Michael Sun edited comment on SOLR-10317 at 5/22/17 11:29 PM:
--------------------------------------------------------------
[~ichattopadhyaya] Hope you get better soon.
[[email protected]] A lot of good progress! For your question about
variation, one suggestion is to report number of segments of the index created
in test. The throughput can vary due to background merging activities. You can
get number of segments using Solr admin/luke endpoint and verify the
relationship.
The metric lib is great. One thing to keep in mind is the metric Dimensionality
issue once all metrics are put into one datastore. For example, for index
throughput, to specify index throughput for test with one shard or two shards,
the only way is to add description in metric name. It can lead to long and hard
to consume metric names, such as
index.throughput.shard-1.replica-1.autocommit-30... There is a good general
description in this netflix blog
(https://medium.com/netflix-techblog/introducing-atlas-netflixs-primary-telemetry-platform-bd31f4d8ed9a,
search Dimensionality).
was (Author: michael.sun):
[~ichattopadhyaya] Hope you get better soon.
[[email protected]] A lot of good progress! For your question about
variation, one suggestion is to report number of segments of the index created
in test. The throughput can vary due to background merging activities. You can
get number of segments using Solr admin/luke endpoint and verify the
relationship.
The metric lib is great. One thing to keep in mind is the Dimensionality issue
once there are many metrics down the road. It can lead to long and hard to
consume metric names, such as
index.throughput.shard-1.replica-1.autocommit-30... There is a good description
in this netflix blog
(https://medium.com/netflix-techblog/introducing-atlas-netflixs-primary-telemetry-platform-bd31f4d8ed9a,
search Dimensionality).
> Solr Nightly Benchmarks
> -----------------------
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
> Issue Type: Task
> Reporter: Ishan Chattopadhyaya
> Labels: gsoc2017, mentor
> Attachments: changes-lucene-20160907.json,
> changes-solr-20160907.json, managed-schema,
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx,
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf,
> solrconfig.xml
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr
> nodes, both in SolrCloud and standalone mode, and record timing information
> of various operations like indexing, querying, faceting, grouping,
> replication etc.
> # It should be possible to run them either as an independent suite or as a
> Jenkins job, and we should be able to report timings as graphs (Jenkins has
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr
> in some of these frameworks above. However, the benchmarks run are very
> limited. Any of these can be a starting point, or a new framework can as well
> be used. The motivation is to be able to cover every functionality of Solr
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure
> [~shalinmangar] and [[email protected]] would help here.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
