On Sat, 11 Sep 2021 15:15:16 +0200 Ichthyostega <p...@ichthyostega.de> wrote:
when we run a testcase, we get a timing measurement as by-product, and
this sparks our hopes for a convenient safety net, allowing to spot
consequences of ongoing code changes early.
Am 11.09.21 um 17:45 schrieb Will Godfrey:
Should we perhaps first define out reference conditions as far as possible.
My own broad-brush 'best' performance is with a sample rate of 48kHz, and
buffer size of either 64 or 128 frames....
Since the Testrunner launches Yoshimi (or in a later planned version loads
the LV2 plugin), I take it for granted that all setup parameters are controlled
by the test. That is, the samplerate, the buffer size, the oscillator size(s).
Obviously this is something on my TODO list. Right now Yoshimi starts with
whatever happens to be the local default configuration. The intention was
to have default parameters somewhere in the test suite; individual tests
might then override them, e.g. to maybe document/cover those nasty known
variations happening for some extreme values....
The test suite is recording these samples. Does it actually matter if one
processor performs a background task between steps 20 and 21, while another
does the same between 40 and 41, or even does two different tasks?
What matters is when we change the code and some features start to take
more computing time *on average*. We want an alarm then. But if some
test just randomly takes a bit more of running time due how the system
handles its processing, then we do not want a "false positive" (i.e.
we do not want an alarm then). So yes, that's a matter of statistics.
Surely the test will remain the same, although if the processor was
especially over-stressed there may be audible errors in how that recording
is presented.
This is what makes Kristian's proposal to integrate LV2 especially enticing.
Using LV2, we could feed a complex MIDI to deliberately create a high load
and then measure the throughput. Thats why I want to follow-up on that
proposal eventually; right now I am focussed on getting the most relevant
basic features of the Testrunner into usable shape, so that we can start
building up actual test cases.
-- Hermann
_______________________________________________
Yoshimi-devel mailing list
Yoshimi-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/yoshimi-devel
