Hi Vlad, Thanks for this initiative!
I’m not a Github Actions guru, but I remember that restarting jobs automatically could be a security risk because the executor might have different permissions. We should probably look into this. Also, I like the idea of restarting jobs if INFRA issues cause problems, but I’m not so keen on restarting flaky tests. It would be better to figure out what’s causing the flakiness and fix it. Would you be interested in exploring this further and figuring out how to make it safe and secure? Regards, Adam Saghy > On Sep 7, 2026, at 12:49 PM, Vlad <[email protected]> wrote: > > Hi all, > > We've been seeing intermittent test failures in CI that don't appear to be > caused by actual code defects, but rather by non-deterministic factors such > as timing, network issues, resource contention, or test ordering. These flaky > failures block builds and require manual re-runs, which wastes CI time and > reviewer attention. > > **Proposal:** Introduce automatic retries for failed tests in CI. For > example, if a test fails, it could be re-run up to two times before the build > is marked as failed. This is a common approach for large test suites and can > help absorb environmental flakiness without masking consistently reproducible > failures. > > Before moving forward, I think we should align on a few points: > > * **Build status:** If a test fails initially but passes on retry, should the > build remain green, or should we still flag it with a warning so we can > identify and address flaky tests? > * **Retry limit:** How many retries should we allow? Should we also define a > limit on the total number of failures before stopping retries, since a high > number of failures is more likely to indicate a real regression? > * **Visibility:** Should we track which tests are being retried over time? > This would help us identify chronically flaky tests instead of allowing them > to remain flaky indefinitely. > > I've already looked into a low-effort way to implement this using our > existing Gradle-based build setup. Once we agree on the approach, I should be > able to prepare a PR quickly. > > Looking forward to your thoughts.
