My original writeup here missed one important detail that Uwe recently clarified for me on the lucene lists. [1]. The constant presence of some Solr jobs in the Jenkins build-queue isn't accidental, it's by design(!) We've historically triggered some jobs "hourly" instead of "nightly" in order to get the maximum number of test-runs / randomization-seeds out of our hardware.
This doesn't eliminate the need for the deduplication I mentioned originally, though it would change a bit how we go about that, since it sounds like it still makes sense to run our tests (and only our tests) in as tight a loop as possible. My updated suggestion would be to: 1. Switch the Solr-reference-guide jobs to be built "daily" instead of the current "hourly". 2. Split the 'Solr-Check' job into two jobs: a "daily" job running static analysis targets, and an "hourly" job running only the tests 3. (Optional) Split 'integrationTests' into their own job, since they're somewhat time consuming and don't use randomization and probably don't benefit much from being run "hourly" This would leave a single job responsible for testing that is triggered "hourly", with all other jobs being run "daily". Any objections to this revised plan? I'll assume lazy-consensus and make the changes sometime next week. Best, Jason [1] https://lists.apache.org/thread/t2ot2l2nyyrjv8q676ok757b7hohqsy5 On Tue, Oct 15, 2024 at 12:40 PM Jan Høydahl <jan....@cominvent.com> wrote: > > +1. cleanups always good. Also, do we need to keep the smoke test 9.7 running > if we don’t think there will be a 9.7.1 release? > > Jan Høydahl > > > On 15 Oct 2024, at 14:32, David Smiley <dsmi...@apache.org> wrote: > > > > +1 Sure; seems like a simple solution. > > > >> On Tue, Oct 15, 2024 at 7:27 AM Jason Gerlowski <gerlowsk...@gmail.com> > >> wrote: > >> > >> Hi all, > >> > >> While cleaning up our Jenkins agents, I noticed that we trigger a lot > >> of long-running (> 1 hr) jobs on a nightly basis. Each family of jobs > >> differs slightly in the gradle task it runs, but there's a lot of > >> overlap. e.g. our 'Solr-Check', 'Solr-Smoketest', and > >> 'Solr-NightlyTests' builds all run our test suite. Combine this with > >> the limited Jenkins hardware we have access to, and it's a recipe for > >> contention. > >> > >> Nightly jobs alone keep our Jenkins agents pegged for at least half > >> the day! This makes us a bad "neighbor" on these boxes. It also > >> prevents us from firing off ad-hoc jobs or doing anything more > >> inventive or rigorous (like multi-JVMs testing) etc. > >> > >> What do you all think of deduplicating the work done by our Jenkins > >> setup? I'm not attached to any particular breakdown, but one example > >> would be to explicitly run 'test' in the 'NightlyTests' job and then > >> exclude the task from running as a part of the 'Check' and 'Smoketest' > >> jobs. > >> > >> Best, > >> > >> Jason > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > >> For additional commands, e-mail: dev-h...@solr.apache.org > >> > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > For additional commands, e-mail: dev-h...@solr.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org For additional commands, e-mail: dev-h...@solr.apache.org
