Hi, We have good steam in the solr-orbit repositories. But some more work to do before we have a stable product. Feel free to jump in and test out the tool and work on open issues.
In particular I need a review on https://issues.apache.org/jira/browse/SOLR-18259 which is the IP clearance that we as a PMC does ourselves to make sure the codebase is good wrt copyright and licensing. I have filled in a form, but need others to review so we can file it with the incubator for official record keeping. Jan > 22. mai 2026 kl. 14:28 skrev Jan Høydahl <[email protected]>: > > Both the solr-orbit and solr-orbit-workloads repositories are created and > initialized with WIP code. > > I have enabled GH issues for both repos and added some initial issues > - https://github.com/apache/solr-orbit/issues > - https://github.com/apache/solr-orbit-workloads/issues > > The ubmrella incubation JIRA is > https://issues.apache.org/jira/browse/SOLR-18253 > > Feel free to jump in and contribute anywhere you like. > > Jan > >> 21. mai 2026 kl. 19:58 skrev Jan Høydahl <[email protected]>: >> >> The VOTE thread has passed. >> >> I'll continue provisioning the solr-orbit and solr-orbit-workloads git >> repositories and push the baseline code. >> We can then create a JIRA umbrella and sub tasks for all the other tasks >> needed. >> >> Jan >> >>> 1. mai 2026 kl. 15:37 skrev Eric Pugh <[email protected]>: >>> >>> I want to share with the community that earlier this week I had the >>> opportunity to visit Kevin and team and talk about Solr Orbit fitting thier >>> specific use case, and I think we have consensus that this solution is how >>> we want to move forward. I know it will solve my client's needs as well. >>> >>> I checked in with Jan, and I think the next step is to get a vote thread on >>> bringing in this codebase as a `solr-orbit` subproject. >>> >>> On 2026/04/23 20:42:30 Eric Pugh wrote: >>>> Home after some travel... >>>> >>>> I think the solr-benchmark-game needs a bit more marinating to see the >>>> true value. Also, Kevin and (???) did some great analysis on it and >>>> pointed out some big weaknesses to it's approach. >>>> >>>> I'm onboard with the solr-orbit plan, and can see how I would use it with >>>> my client to establish the benchmarks needed. >>>> >>>> On 2026/04/17 20:38:17 "Kevin Liang (BLOOMBERG/ 919 3RD A) via dev" wrote: >>>>> "Solr Orbit" seems a fine name to me. >>>>> >>>>> Re: David's point on picking one of search-benchmark-game vs. >>>>> solr-benchmark-fork >>>>> >>>>> I'll add my opinion as I've used / made code changes to both and have >>>>> provided this feedback to Eric / Jan. >>>>> >>>>> The search-benchmark-game is nice for its simplicity and results >>>>> presentation. It can quickly give you some data points between two >>>>> versions of software. That being said, at best it seems geared more >>>>> towards assessing a library (lucene) than a full distributed system >>>>> (solr). Its feature set is limited and the methodology for executing >>>>> queries / drawing statistical conclusions leaves something to be desired >>>>> (can expand in detail if anyone wants to hear). >>>>> >>>>> In contrast, solr-benchmark (OSB fork) is building off a feature set >>>>> constructed over the better part of a decade. It's a much larger codebase >>>>> and a more mature benchmarking framework with much greater capability to >>>>> do different kinds of performance test workloads. >>>>> >>>>> Personally I would go with solr-benchmark. Building a web interface on >>>>> top (or hooking up a jenkins build pipeline) can be done. And like we >>>>> said in the meeting, just getting people to run the tool independently >>>>> and publishing their results, even as plain text on the wiki, would be a >>>>> good first step win. >>>>> >>>>> From: [email protected] At: 04/16/26 13:00:14 UTC-4:00To: >>>>> [email protected] >>>>> Subject: Re: Solr Performance Benchmarking Discussion Page >>>>> >>>>> It's not work using Elastic Inc's trademark "Rally" if we can avoid it. >>>>> They >>>>> are known to pursue such violations. And I believe it is a violation of >>>>> the >>>>> Apache license too. I like "Solr Orbit" :) >>>>> >>>>> I'll wait some time to let all voices be heard wrt the various tools and >>>>> which >>>>> to pick going forward, then if everyone rallies (pun intended) around >>>>> solr-benchmark, then we can proceed. >>>>> >>>>> Jan >>>>> >>>>>> 16. apr. 2026 kl. 14:36 skrev David Smiley <[email protected]>: >>>>>> >>>>>> Why avoid the word "Rally" if that's its foundation/lineage? If we >>>>>> want to avoid risk of using that word, then we simply ask >>>>>> ElasticSearch with our proposed name for our fork that includes that >>>>>> word. >>>>>> >>>>>> On Thu, Apr 16, 2026 at 3:37 AM Jan Høydahl <[email protected]> >>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Thanks for presenting the solr-benchmark tool on the community meetup >>>>> yesterday Kevin. >>>>>>> I'll encourage others to take it for a spin. There are rough edges but >>>>>>> you >>>>> should be able to get some results. >>>>>>> >>>>>>> We also discussed the potential for moving the tool from my github >>>>>>> space to >>>>> asf. >>>>>>> That would make it easier for the community to contribute and >>>>>>> collaborate. >>>>> But it requires 2-3 PMC members interested in being maintainers. >>>>>>> If we make it an asf repo, David noted that we are not required to >>>>>>> publish >>>>> official releases until we feel compelled to do so. >>>>>>> Also, David noted the naming collision with the internal solr-benchmark >>>>> module. How about "solr-orbit", which is a homage to "Rally" (orbiting >>>>> the race >>>>> track) but in our Solar-system universe :) >>>>>>> >>>>>>> As a next step I'll start a VOTE thread for accepting the two repos into >>>>> ASF. The results of the VOTE will guide further action. >>>>>>> >>>>>>> Jan >>>>>>> >>>>>>>> 8. mars 2026 kl. 20:42 skrev Jan Høydahl <[email protected]>: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I read through the benchmarking wiki page today. It was well written, >>>>> thanks Kevin for reviving this effort, and in such a structured way! >>>>>>>> >>>>>>>> There's a renewed energy around benchmarking, and more tooling options >>>>>>>> than >>>>> ever. So to add to the mix I'm presenting yet another one 🤣🤣, "Solr >>>>> Benchmark": >>>>>>>> >>>>>>>> https://github.com/janhoy/solr-benchmark >>>>>>>> >>>>>>>> You may quickly notice that this looks familiar. And yes, it is a >>>>>>>> fork/port >>>>> of Rally/OpensearchBenchmark, ported to provision and benchmark Solr >>>>> clusters, >>>>> using the same datasets and "workload"s as those tools. >>>>>>>> I first tried to start a fork 2 or 3 years ago but it stranded. I made >>>>>>>> a >>>>> new effort using LLM agents and this first working version was prepared >>>>> in a >>>>> few afternoons. >>>>>>>> Even if the foundation is solid and proven over many years, this >>>>>>>> initial >>>>> port is not complete, view it as a MVP and WIP. Only one workload / >>>>> dataset is >>>>> ported so far. Take it for a spin... >>>>>>>> >>>>>>>> I'll defer to Kevin to add it to the tools list of the wiki and >>>>>>>> continue >>>>> the analysis effort with this as one of the contenders. >>>>>>>> >>>>>>>> Jan >>>>>>>> >>>>>>>>> 2. mars 2026 kl. 17:13 skrev Kevin Liang (BLOOMBERG/ 919 3RD A) >>>>> <[email protected]>: >>>>>>>>> >>>>>>>>> Hello everyone, >>>>>>>>> >>>>>>>>> Given the recent interest and discussion around Solr performance >>>>> benchmarking, I figured it would be useful to 1) centralize the >>>>> discussion and >>>>> 2) bring it to a long-lived format (that's not email). So with that, I >>>>> have >>>>> started >>>>> https://cwiki.apache.org/confluence/display/SOLR/Solr+Performance+Benchmarking >>>>> >>>>> (I figured there's still more discussion to be had before it becomes a >>>>> SIP with >>>>> technical requirements). >>>>>>>>> >>>>>>>>> I encourage anyone and everyone who is interested to provide their >>>>>>>>> input >>>>> (comment or edit). This is a community initiative, and shouldn't be >>>>> limited by >>>>> me or any biases I may have. Hopefully people find this useful in moving >>>>> the >>>>> discussion forward. >>>>>>>>> >>>>>>>>> -Kevin >>>>>>>> >>>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>>>> >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
