The new GitHub Actions workflows that run Java and Python tests against different targets (macos, ubuntu, windows) are great! But just like our Jenkins infra they flake occasionally. Should we be re-running all of these jobs until we get green runs?
Unfortunately it's not possible to re-run an individual job in a workflow [1], the only option is to re-run all jobs, so flaky tests become even more problematic. I see two options: 1) Consider it "good enough" if just Jenkins CI passes and any GitHub actions failures appear to be flakes. 2) Require that all Jenkins and GitHub checks pass. My vote is for (2). (1) risks merging legitimate breakages, and one could argue that making flaky tests extra painful is a good thing. Also we can always make an exception if an obvious flake is blocking a critical PR. Also FYI - at first I thought these workflows only had the stdout available, but the test report directory is also zipped and uploaded as an artifact. When a failure occurs you can download it to get the full output: [image: image.png] Brian [1] https://github.community/t/ability-to-rerun-just-a-single-job-in-a-workflow/17234
