janhoy opened a new issue, #99: URL: https://github.com/apache/solr-orbit/issues/99
### Is your feature request related to a problem? Please describe Apache Solr's performance story is currently anecdotal. Regressions get found by users in production, months after the commit that caused them, when the bisect range is hundreds of commits wide and whoever wrote the change has moved on. We want a public page that answers, at a glance, the question nobody can answer today: **is Solr getting faster or slower, and when did it change?** Per workload, how indexing throughput and query latency have moved across Solr versions — and from a suspicious dip, click-through to the full report behind it: the exact Solr commit, JVM build, hardware, heap, GC, shard layout, and workload revision that produced it. The model is [Lucene Nightly Benchmarks](https://benchmarks.mikemccandless.com), which has quietly been one of the most valuable artifacts in the Lucene project for over a decade. <img width="1490" height="986" alt="Image" src="https://github.com/user-attachments/assets/ced4bf7f-8111-4672-905e-7cf235eb170c"; /> ### Describe the solution you'd like `reportr`: a Python CLI bundled in this repo that reads the enriched JSON reports written by `launchr` (#98) and emits a **static** site — no backend, no database, no runtime dependency. The SPA shell ships inside reportr itself, so updating the tool updates the site's frontend; charts are Apache ECharts, vendored rather than CDN-loaded. Deployed to `solr.apache.org/benchmarks/`. ``` reportr build --local ./benchmark-results --serve # local development, charts at localhost:8080 reportr build --config reportr.yaml # production, reads from the bucket reportr build --full # ignore state, reprocess everything ``` Four choices worth agreeing on before the PRs: - **The x-axis is the version timeline, not the run date.** Points sit at the date of the commit or release tag they measure, so re-benchmarking an old version lands in its correct historical position, and `main` and `branch_10x` are separate lines. A line is identified by version kind, branch, hardware profile *and* cluster config — so the site never silently compares a 3-node EC2 result against a laptop run. - **Annotations are curated by hand** in an `annotations.yaml` ("upgraded to r7iz.4xlarge", "SOLR-12345 facet regression"). No automated regression detection on the site: a human notices the dip, investigates, and writes down what they found. - **Builds are incremental**, so a nightly run appends rather than reprocessing years of history. An incremental build and a `--full` rebuild must produce identical output. - **Static per-workload JSON data files** as the first data backend. Good for roughly the first couple of thousand runs; the escape route is Parquet plus DuckDB at build time, which does not change the website. First PR is small and local: read reports from a directory, emit per-workload JSON, render one trend chart with click-through, serve on localhost — iterable on a laptop against real `launchr` output. Retention, annotation compilation, the dashboard overview, and automated deployment follow. ### Describe alternatives you've considered Using some pre-made reporting application ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
