Hi Will,
Hi Kristian,
just a status update.
The Testrunner now passes a default-state file to Yoshimi,
which hopefully settles most issues with different default
settings by the user.
Next I started to investigate those strange differences we saw
when running on different CPUs. As said, the suspicion was that
what we see here is just the inherent imprecision in floating
point calculations. But, as always, I want to verify this
theory (since it could also be due to some hidden bug).
I'm not finished with that investigation yet. Basically what
I want to do is to watch the calculations with the debugger,
to narrow down the point where the numbers start to differ.
However, my first attempt ended with a surprise...
Starting point was that I didn't have debugging informations
on my Laptop, since the Ubuntu PPA builder does not build a
debug info package (as the debian build does). And, moreover,
debugging an optimised release build is always a bit tricky.
Thus I set up a development branch on top of current master
and configured the Testrunner to use this debug-build-yoshimi.
And, guess what...
- the calculated sound differs from the release build!!
- and the debug build produces 100% the same sound on
both my desktop and the Laptop
These differences are also quite small (I can't hear them),
but numerically they are larger than the differences I saw
between the two release builds (-84dB RMS vs -110dB RMS)
So seemingly Optimisation also plays some role here
Generally speaking, this doesn't invalidate the concept
of test coverage; we just have to figure out the proper
tolerance level. I still want to dig a bit deeper, until
I can see that in fact we're dealing with rounding
problems here. But beyond that, the Testruner is more
or less ready for use now. Likely that we'll add further
features when needed, like e.g. a "filter" to run only
some part of the Testsuite or maybe only a single test.
-- Hermann
_______________________________________________
Yoshimi-devel mailing list
Yoshimi-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/yoshimi-devel
