Coming to this late, but a summary email of the "state of CI" is an interesting idea.
We used to have something similar, for folks that remember Erick Erickson's "BadApple" summary emails. If I remember right, those got quite long because of the test flakiness going on at the time. My (possibly incorrect) impression was that folks started tuning out the summaries, even though there was a lot of value there in terms of surfacing new failures that might've been related to ongoing work. An LLM seems like it'd do a good job of bringing down the manual work required but that'd make it hard to run as a CI job. (Who would pay for the tokens, etc.?) Anyway, just thinking aloud a bit about the questions that'd come up if this was pursued, in case anyone else has answers that'd help move this forward. Best, Jason On Tue, May 12, 2026 at 8:25 PM David Smiley <[email protected]> wrote: > > Ah; thanks Gus (And Jan!) > > On Tue, May 12, 2026 at 6:45 PM Gus Heck <[email protected]> wrote: > > > Not entirely unnoticed https://issues.apache.org/jira/browse/SOLR-18203 > > > > On Tue, May 12, 2026 at 11:01 AM David Smiley <[email protected]> wrote: > > > > > It appears the job for the "nightly" tests haven't passed since December > > > 15th: > > > https://ci-builds.apache.org/job/Solr/job/Solr-NightlyTests-main/ > > > with ~4 tests failing on average. Confusingly all 4 are the same, > > > according to the most recent failing job run: > > > org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test > > > Not sure why the same one repeats 4 times; I don't see @Repeat > > > > > > Any way... it's troubling this can apparently go on unnoticed for so > > long. > > > > > > I've been thinking, it'd be amazing to have a weekly "state of CI" > > summary > > > report sent to the dev list. The information is out there, and some of > > it > > > even summarized like on fucit. Such a report could be a new Jenkins job > > > that runs weekly and emails the dev list, not the build list that we > > mostly > > > ignore. > > > > > > ~ David Smiley > > > Apache Lucene/Solr Search Developer > > > http://www.linkedin.com/in/davidwsmiley > > > > > > > > > -- > > http://www.needhamsoftware.com (work) > > https://a.co/d/b2sZLD9 (my fantasy fiction book) > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
