The VOTE thread has passed. I'll continue provisioning the solr-orbit and solr-orbit-workloads git repositories and push the baseline code. We can then create a JIRA umbrella and sub tasks for all the other tasks needed.
Jan > 1. mai 2026 kl. 15:37 skrev Eric Pugh <[email protected]>: > > I want to share with the community that earlier this week I had the > opportunity to visit Kevin and team and talk about Solr Orbit fitting thier > specific use case, and I think we have consensus that this solution is how we > want to move forward. I know it will solve my client's needs as well. > > I checked in with Jan, and I think the next step is to get a vote thread on > bringing in this codebase as a `solr-orbit` subproject. > > On 2026/04/23 20:42:30 Eric Pugh wrote: >> Home after some travel... >> >> I think the solr-benchmark-game needs a bit more marinating to see the true >> value. Also, Kevin and (???) did some great analysis on it and pointed out >> some big weaknesses to it's approach. >> >> I'm onboard with the solr-orbit plan, and can see how I would use it with my >> client to establish the benchmarks needed. >> >> On 2026/04/17 20:38:17 "Kevin Liang (BLOOMBERG/ 919 3RD A) via dev" wrote: >>> "Solr Orbit" seems a fine name to me. >>> >>> Re: David's point on picking one of search-benchmark-game vs. >>> solr-benchmark-fork >>> >>> I'll add my opinion as I've used / made code changes to both and have >>> provided this feedback to Eric / Jan. >>> >>> The search-benchmark-game is nice for its simplicity and results >>> presentation. It can quickly give you some data points between two versions >>> of software. That being said, at best it seems geared more towards >>> assessing a library (lucene) than a full distributed system (solr). Its >>> feature set is limited and the methodology for executing queries / drawing >>> statistical conclusions leaves something to be desired (can expand in >>> detail if anyone wants to hear). >>> >>> In contrast, solr-benchmark (OSB fork) is building off a feature set >>> constructed over the better part of a decade. It's a much larger codebase >>> and a more mature benchmarking framework with much greater capability to do >>> different kinds of performance test workloads. >>> >>> Personally I would go with solr-benchmark. Building a web interface on top >>> (or hooking up a jenkins build pipeline) can be done. And like we said in >>> the meeting, just getting people to run the tool independently and >>> publishing their results, even as plain text on the wiki, would be a good >>> first step win. >>> >>> From: [email protected] At: 04/16/26 13:00:14 UTC-4:00To: >>> [email protected] >>> Subject: Re: Solr Performance Benchmarking Discussion Page >>> >>> It's not work using Elastic Inc's trademark "Rally" if we can avoid it. >>> They >>> are known to pursue such violations. And I believe it is a violation of the >>> Apache license too. I like "Solr Orbit" :) >>> >>> I'll wait some time to let all voices be heard wrt the various tools and >>> which >>> to pick going forward, then if everyone rallies (pun intended) around >>> solr-benchmark, then we can proceed. >>> >>> Jan >>> >>>> 16. apr. 2026 kl. 14:36 skrev David Smiley <[email protected]>: >>>> >>>> Why avoid the word "Rally" if that's its foundation/lineage? If we >>>> want to avoid risk of using that word, then we simply ask >>>> ElasticSearch with our proposed name for our fork that includes that >>>> word. >>>> >>>> On Thu, Apr 16, 2026 at 3:37 AM Jan Høydahl <[email protected]> wrote: >>>>> >>>>> Hi, >>>>> >>>>> Thanks for presenting the solr-benchmark tool on the community meetup >>> yesterday Kevin. >>>>> I'll encourage others to take it for a spin. There are rough edges but you >>> should be able to get some results. >>>>> >>>>> We also discussed the potential for moving the tool from my github space >>>>> to >>> asf. >>>>> That would make it easier for the community to contribute and collaborate. >>> But it requires 2-3 PMC members interested in being maintainers. >>>>> If we make it an asf repo, David noted that we are not required to publish >>> official releases until we feel compelled to do so. >>>>> Also, David noted the naming collision with the internal solr-benchmark >>> module. How about "solr-orbit", which is a homage to "Rally" (orbiting the >>> race >>> track) but in our Solar-system universe :) >>>>> >>>>> As a next step I'll start a VOTE thread for accepting the two repos into >>> ASF. The results of the VOTE will guide further action. >>>>> >>>>> Jan >>>>> >>>>>> 8. mars 2026 kl. 20:42 skrev Jan Høydahl <[email protected]>: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I read through the benchmarking wiki page today. It was well written, >>> thanks Kevin for reviving this effort, and in such a structured way! >>>>>> >>>>>> There's a renewed energy around benchmarking, and more tooling options >>>>>> than >>> ever. So to add to the mix I'm presenting yet another one 🤣🤣, "Solr >>> Benchmark": >>>>>> >>>>>> https://github.com/janhoy/solr-benchmark >>>>>> >>>>>> You may quickly notice that this looks familiar. And yes, it is a >>>>>> fork/port >>> of Rally/OpensearchBenchmark, ported to provision and benchmark Solr >>> clusters, >>> using the same datasets and "workload"s as those tools. >>>>>> I first tried to start a fork 2 or 3 years ago but it stranded. I made a >>> new effort using LLM agents and this first working version was prepared in >>> a >>> few afternoons. >>>>>> Even if the foundation is solid and proven over many years, this initial >>> port is not complete, view it as a MVP and WIP. Only one workload / dataset >>> is >>> ported so far. Take it for a spin... >>>>>> >>>>>> I'll defer to Kevin to add it to the tools list of the wiki and continue >>> the analysis effort with this as one of the contenders. >>>>>> >>>>>> Jan >>>>>> >>>>>>> 2. mars 2026 kl. 17:13 skrev Kevin Liang (BLOOMBERG/ 919 3RD A) >>> <[email protected]>: >>>>>>> >>>>>>> Hello everyone, >>>>>>> >>>>>>> Given the recent interest and discussion around Solr performance >>> benchmarking, I figured it would be useful to 1) centralize the discussion >>> and >>> 2) bring it to a long-lived format (that's not email). So with that, I have >>> started >>> https://cwiki.apache.org/confluence/display/SOLR/Solr+Performance+Benchmarking >>> >>> (I figured there's still more discussion to be had before it becomes a SIP >>> with >>> technical requirements). >>>>>>> >>>>>>> I encourage anyone and everyone who is interested to provide their input >>> (comment or edit). This is a community initiative, and shouldn't be limited >>> by >>> me or any biases I may have. Hopefully people find this useful in moving >>> the >>> discussion forward. >>>>>>> >>>>>>> -Kevin >>>>>> >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
