Yesterday I committed several fixes to the simulation package, which should improve reliability of these tests. If these failures still persist we should BadApple the ones that keep failing.
> On 5 Jun 2018, at 10:14, Adrien Grand <jpou...@gmail.com> wrote: > > Thanks Steve. IndexSizeTriggerTest seems to be the test that fails most often > on this build indeed. > > Le lun. 4 juin 2018 à 19:30, Steve Rowe <sar...@gmail.com > <mailto:sar...@gmail.com>> a écrit : > I looked at the way that tests are run, an tthe only difference I see in the > smoke tester jobs is that tests are run twice, once each for Java8 and Java9. > Compared to non-smoke-tester jobs, this will double the likelihood of > overall failure. > > I looked at the suites that failed in the last ten runs on those two smoke > tester jobs. Except for SearchHandlerTest, which I have since (hopefully) > fixed, there are seven suites with failed tests - here they are along with > their packages: > > TestExecutePlanAction o.a.s.cloud.autoscaling.sim > TestComputePlanAction o.a.s.cloud.autoscaling.sim > TestTriggerIntegration o.a.s.cloud.autoscaling.sim > IndexSizeTriggerTest o.a.s.cloud.autoscaling > CreateRoutedAliasTest o.a.s.cloud > ReplaceNodeTest o.a.s.cloud > MetricsHistoryHandlerTest o.a.s.handler.admin > > Some of those are pretty regular offenders AFAICT from > http://fucit.org/solr-jenkins-reports/failure-report.html > <http://fucit.org/solr-jenkins-reports/failure-report.html> . > > Andrzej Białecki did some work on IndexSizeTriggerTest (SOLR-12392) and > un-bad-apple’d its tests, but at least one of them is still failing since > then - I’ll go add a comment on the issue. > > -- > Steve > www.lucidworks.com <http://www.lucidworks.com/> > > > On Jun 4, 2018, at 11:16 AM, Erick Erickson <erickerick...@gmail.com > > <mailto:erickerick...@gmail.com>> wrote: > > > > Adrien: > > > > "Do you know whether there is something that makes these jobs any > > different?" > > > > Unfortunately no. I'm not really very well versed in the various test > > environments, maybe Uwe or Steve Rowe or Hoss might have some insight? > > > > On Mon, Jun 4, 2018 at 3:34 AM, Adrien Grand <jpou...@gmail.com > > <mailto:jpou...@gmail.com>> wrote: > >> Thanks for helping on this front Erick. I noticed a significant decrease in > >> noise since you started badapple-ing bad tests, but I'm observing that our > >> smoke-release builds still keep failing because of Solr tests (10 out of > >> the > >> last 10 builds) in spite of the fact that they disable bad apples: > >> - https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/ > >> <https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/> > >> - https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/ > >> <https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/> > >> > >> Do you know whether there is something that makes these jobs any different? > >> > >> Le mar. 29 mai 2018 à 18:12, Erick Erickson <erickerick...@gmail.com > >> <mailto:erickerick...@gmail.com>> a > >> écrit : > >>> > >>> With the long weekend and the fact that the number of non-BadApple > >>> tests is fairly small this week, I'll skip adding more BadApple tests > >>> until next week. > >>> > >>> We're scarily close to the non-BaApple'd tests coming under control. > >>> If we're lucky, we can draw a line in the sand soon then start working > >>> on the backlog. I'll be encouraged if we can start shrinking the > >>> BadApple'd tests. > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > >>> <mailto:dev-unsubscr...@lucene.apache.org> > >>> For additional commands, e-mail: dev-h...@lucene.apache.org > >>> <mailto:dev-h...@lucene.apache.org> > >>> > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > <mailto:dev-unsubscr...@lucene.apache.org> > > For additional commands, e-mail: dev-h...@lucene.apache.org > > <mailto:dev-h...@lucene.apache.org> > > > > > > Begin forwarded message: > > > From: Erick Erickson <erickerick...@gmail.com > > <mailto:erickerick...@gmail.com>> > > Subject: Re: Not adding badapples this week. > > Date: June 4, 2018 at 11:16:56 AM EDT > > To: dev@lucene.apache.org <mailto:dev@lucene.apache.org> > > Reply-To: dev@lucene.apache.org <mailto:dev@lucene.apache.org> > > > > Adrien: > > > > "Do you know whether there is something that makes these jobs any > > different?" > > > > Unfortunately no. I'm not really very well versed in the various test > > environments, maybe Uwe or Steve Rowe or Hoss might have some insight? > > > > On Mon, Jun 4, 2018 at 3:34 AM, Adrien Grand <jpou...@gmail.com > > <mailto:jpou...@gmail.com>> wrote: > >> Thanks for helping on this front Erick. I noticed a significant decrease in > >> noise since you started badapple-ing bad tests, but I'm observing that our > >> smoke-release builds still keep failing because of Solr tests (10 out of > >> the > >> last 10 builds) in spite of the fact that they disable bad apples: > >> - https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/ > >> <https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/> > >> - https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/ > >> <https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/> > >> > >> Do you know whether there is something that makes these jobs any different? > >> > >> Le mar. 29 mai 2018 à 18:12, Erick Erickson <erickerick...@gmail.com > >> <mailto:erickerick...@gmail.com>> a > >> écrit : > >>> > >>> With the long weekend and the fact that the number of non-BadApple > >>> tests is fairly small this week, I'll skip adding more BadApple tests > >>> until next week. > >>> > >>> We're scarily close to the non-BaApple'd tests coming under control. > >>> If we're lucky, we can draw a line in the sand soon then start working > >>> on the backlog. I'll be encouraged if we can start shrinking the > >>> BadApple'd tests. > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > >>> <mailto:dev-unsubscr...@lucene.apache.org> > >>> For additional commands, e-mail: dev-h...@lucene.apache.org > >>> <mailto:dev-h...@lucene.apache.org> > >>> > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > <mailto:dev-unsubscr...@lucene.apache.org> > > For additional commands, e-mail: dev-h...@lucene.apache.org > > <mailto:dev-h...@lucene.apache.org> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > <mailto:dev-unsubscr...@lucene.apache.org> > For additional commands, e-mail: dev-h...@lucene.apache.org > <mailto:dev-h...@lucene.apache.org> >
