> > After a few givebacks, official architectures all pass (minus riscv64 the > > buildds of which are too slow to not have timeouts, so I skipped tests on it > > in -3). This further confirms my suspicion that the failures were indeed due > > to concurrency problems. > > Well, sorry but I don't think downgrading this bug is ok. Flaky tests > are RC since trixie. See my build history, on AWS instances of type > c7a.large and m7a.large (both amd64), which incidentally have 2 CPUs, > nothing special:
Note that - the (latent) flaky issues seem known and have existed for many years. - upstream is working on resolving them (https://github.com/fish-shell/fish-shell/issues/11815); I recommend not spending too much time on this, since it's really an upstream problem. - upstream does not run some of the flaky tests in CI, based on CI or FISH_CI_SAN environment variables. For example, the failing "fkr.py" # Disable under CI - keeps failing because the timing is too tight if "CI" in os.environ: sys.exit(127) However, they all pass when given enough resources and/or with `export FISH_TEST_MAX_CONCURRENCY=1`, and having them enabled by default can help developers. - So as a workaround, maybe either - set `export CI=1` when running tests - and/or `export FISH_TEST_MAX_CONCURRENCY=1` at cmake-configure time - or disable affected tests until upstream fixes these.
