On 21.11.2021 00:57, Ichthyostega wrote: > Am 20.11.21 um 20:28 schrieb Kristian Amlie: >> With all these unpredictable effects going on, perhaps we should just >> turn >> -ffast-math off and see if that yields something more stable? The >> -fexcess-precision=fast option, which is implied by -ffast-math, >> explicitly >> states that the rounding errors are unpredictable. > > Well, initially I hoped we could get an easy-to-use tool for the > developers, allowing to run those tests frequently. If you need > to make a special build, chances are that the tests will only > be run for release preparation, if at all.
But we can also run them on pull requests on Github, no? I was imagining this would be used as a gate keeper during code submission, so that we don't let bad changes in. We can control this environment very easily, and I think most people don't actually need to run the tests locally unless Github tells them they broke something and need to investigate. At this point they're already invested, and likely to follow setup instructions even if it means recompiling with different options. > Obviously, -O0 would be a candidate for such a build, but that > more or less defeats the usefulness of timing tests. That is the > reason why I am currently still trying to get the results for a > typical release build stabilised enough so that we could at least > ignore minor differences below -80 or -100dB Yeah, we could. But I don't agree that -O0 defeats timing tests. It's true they won't be 100% accurate, but it's the relative difference that counts, and this is easier to measure across compilers when using -O0. And perhaps we don't even need -O0, it could be that turning off the float-specific options could already be enough, which would leave loop optimizations and such in place. > However, if that turns out as impossible, than we'll probably > need to define special build flags for the testsuite; and then > maybe indeed use some of those special flags to get some degree > of optimisation, but no SSE. > > > Incidentally, I'd like to stress the fact that the original differences > are really minute. The nasty beast turns out to be the AnalogFilter, > which kind of amplifies those tiny discrepancies, especially when > they occur on the filter coefficients. > > So another option would be to have several sets of tests, and > focus on some basic tests (without the Filter), which could then > be run routinely, and an extended test suite which requires a > specific build. This could be a solution. We would then relax the tolerance level for those tests, I assume. -- Kristian _______________________________________________ Yoshimi-devel mailing list Yoshimi-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/yoshimi-devel
