Running the tests on every PR raised will make reviewing patches MUCH simpler. Today, on my laptop, it’s 40 minutes to 1 hour to run the full process, and I can’t use my machine during that time.. Being able to look at a PR and see that the tests run will make me more likely to review ;-).
Does the output from Crave get integrated into Github Actions as a “Check”?? > On Jan 26, 2023, at 1:11 AM, Yuvraaj Kelkar <yuvr...@gmail.com> wrote: > > Apologies: I just signed up for this email list right now. I was signed up > for the old Lucene list but didn't realize that the split would impact the > mailing list as well. Explains a lot, now that I think about it. > > I'll answer all the questions here, that I see from the thread: > 1: (Noble) Full test for every PR raised: Ideally yes. The goal is to make > sure that we can provide easy access to the sort of machines that can be used > to really stress test Solr. Providing such "monster machines" only once in a > while (eg once a day because of a crontab) goes against the whole idea of > giving everyone enough infrastructure to have confidence in their own PR. > 2: (Gus) Is Crave really prepared for anyone to come and use them: I'm going > to just say "I sure hope so": We've been able to handle real life concurrent > builds from 200+ developers. We've done synthetic testing to simulate > thousands (4000) of developers and builds. > Also, if the question was: "Can Crave handle the cost of this > infrastructure?": The answer is: Within reason, yes. > We try to reduce the cost of infrastructure by being clever about turning it > on when needed and turning it off when idle. In other words, the duty cycle > determines cost. There are also limits we've defined to ensure that the cost > to Crave does not exceed a monthly budget. > I'm open to suggestions or funding. > > 3. (Gus) Is there a limit per person or project: Not really; do you want us > to put limits? > 4. (Gus) Does it require paid / membership: > Require: No. > There exists a paid version, but it is not required for open source projects. > > 5: (Ishaan) Can multiple developers execute their tests at the same time: > Yes. > > Thanks, > -UV > > On 2023/01/25 23:36:45 Noble Paul wrote: >> This is interesting. >> >> So, if the PR is merged , we will have the full test running on crave.io >> for every PR raised? >> >> On Thu, Jan 26, 2023 at 9:22 AM David Smiley wrote: >> >>> We haven't been running all our tests in GitHub Actions (i.e. PR >>> validation) because it was too time consuming to do so. I don't recall how >>> slow it was when someone last tried; it's probably better now but still >>> slow. To make up for this, there is a GHA only for SolrJ if a PR touches >>> SolrJ. >>> >>> There's now a PR here to introduce a new GHA that builds on Crave.io on a >>> beefy machine: https://github.com/apache/solr/pull/1303 The PR validation >>> took 11 minutes which is similar to the amount of time it took a GHA to >>> just do precommit checks -- 10 minutes :-) >>> I think we can remove the SolrJ specific GHA as it'll be redundant. >>> >>> Furthermore, anyone can use this to run tests from the convenience of your >>> laptop at the CLI while you are in the middle of any change (doesn't matter >>> what you have committed or not, pushed or not). To do so, run: crave run >>> -- './gradlew localSettings && ./gradlew --max-workers=`nproc` >>> -Ptests.jvms=48 test' >>> >>> Yeah that's long. There is a discussion in JIRA underway that may lead to >>> eliminating the "localSettings" step if, for example, it's moved to a bash >>> script executed by the gradle wrapper (my proposal). I should also be able >>> to configure crave with a default run configuration with this baked in. >>> I'll post an update when I'm able to do that. >>> >>> ~ David Smiley >>> Apache Lucene/Solr Search Developer >>> http://www.linkedin.com/in/davidwsmiley >>> >> >> >> -- >> ----------------------------------------------------- >> Noble Paul _______________________ Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
