On Wed, Dec 21, 2016 at 11:04:34PM +0100, Santiago Vila wrote: > So, if establishing a threshold is the only way to achieve that for > example Bug #843038 in "elki" is upgraded to serious again, so be it, > but as I said, it would be a pity if we invest our time trying to > estimate probabilities instead of actually ensuring that packages > build all the time as a shared goal with reproducible builds.
Santiago, I think you're doing important work drawing attention to a class of bugs that has long gone unnoticed. However, you seem to suggest that simply disabling occasionally failing tests will improve the quality of Debian. I'm not so sure about that. Those tests might provide important coverage for high-level functions our users actually care about, so I'd want those tests to get fixed instead of disabled. But timing issues are hard to debug or even reproduce on developers' machines, upstreams may need convincing, while RC-bugs mere weeks before the freeze create enormous time pressure. How can we ensure those tests aren't just disabled and forgotten? Florian