Thank you for clarifying everybody, this is valuable expertise.
On 2019/02/27 09:01:00, Ilya Kasnacheev <i...@gmail.com> wrote: > Hello!> > > We also have this wonderful tool: https://mtcga.gridgain.com/> > > It keeps track of tests which are currently flaky or under investigation,> > and given your change it will report non-flaky tests that failed, skipping> > all problematic tests.> > > It talks to TeamCity so it can run tests for you, check their results and> > comment JIRA with "Bot Visa", see:> > https://issues.apache.org/jira/browse/IGNITE-11299 comment from "Ignite TC> > Bot"> > > Since we don't have too many flaky tests thanks to Make TeamCity Green> > Again movement, it works reasonably well.> > > A year ago we did not have any of this but now we do, and we were able to> > shave 80% of test-checking busy work. This is mostly thanks to Dmitry> > Pavlov.> > > Regards,> > -- > > Ilya Kasnacheev> > > > ср, 27 февр. 2019 г. в 01:39, Stanislav Kozlovski <> > stanislav_kozlov...@outlook.com>:> > > > Hey there Ignite community,> > >> > > I contribute to a fellow open-source project - Apache Kafka - and there we> > > have been fighting flaky tests a lot. We run Java 8 and Java 11 builds on> > > every Pull Request and due to test flakiness, almost all of them turn out> > > red with 1 or 2 tests (completely unrelated to the change in the PR)> > > failing. This has resulted in committers either ignoring them and merging> > > the changes or in the worst case rerunning the hour-long build until it> > > becomes green.> > > This test flakiness has also slowed down our releases significantly.> > >> > > In general, I was just curious to understand if this is a problem that> > > your project faces as well. Does your project have a lot of intermittently> > > failing tests, do you have any active process of addressing such tests> > > (during the initial review, after realizing it is flaky, etc). Any > > pointers> > > will be greatly appreciated!> > >> > > Thanks,> > > Stanislav> > >> > >> > > <http://aka.ms/weboutlook>> > >> >
