I've created a "Random test failure" project that we can use to group these issues: https://github.com/orgs/sagemath/projects/1 A few issues are already added there, feel free to add further ones (either existing or newly created ones).
Moreover, CI now has a more robust handling of files that are known to have randomly failing tests (https://github.com/sagemath/sage/blob/9f905404644a1a58b7808293896b87df80f9587a/.github/workflows/ci-meson.yml#L25). I'm planning to expand the list to include all the issues in the above project in a couple of days. On Saturday, August 30, 2025 at 2:13:42 AM UTC+8 Michael Orlitzky wrote: > On 2025-08-28 20:46:47, '[email protected]' via sage-devel wrote: > > Lines annotated with "known bug" are always skipped. What do you think > > about the following: > > 1. Introduce a new "random failure" tag and use that to annotate > doctests > > that are known to fail sometimes (either with segfaults, timeouts, or > > differing output) > > 2. Normally on CI, skip the execution of such tests. > > 3. In one special run after a release (or perhaps also for PRs?), only > > execute the "random failure" tests. Since there will not be that many > such > > annotated tests, we could even run it a couple of times and only report > an > > error if none of the runs was successful. > > I think this would be counter-productive because it would completely > hide these failures from developers, but leave them there for users to > hit. Not tested / known bug would be better in that regard, since it > hides them from everyone. > > But I really don't think we need a special solution for this. These > tests usually aren't that hard to fix. In the time it will take to > come up with an elaborate way to ignore them, most of them could be > fixed. > -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/sage-devel/51a05f92-2a64-4b6e-899d-49ab57fecf17n%40googlegroups.com.
