On Wed, 26 May 2021 10:53:14 +0200
Kristian Amlie <krist...@amlie.name> wrote:
>On 25.05.2021 17:07, Ichthyostega wrote:
>> Am 17.05.21 um 13:16 schrieb Will Godfrey:
>>> However, I'd still especially like to see the testing work from
>>> Hermann, ...
>>
>>
>> Hi Yoshimi-developers,
>>
>> currently I'm about to find out about ways to hook into Yoshimi for
>> testing.
>> I'd appreciate any further hints, recommendations or proposals.
>>
>> As we've discussed, the first thing I'd like to concentrate on is to allow
>> for automated acceptance tests on the engine level. As you probably know,
>> usually the testing effort today is organised into a "test pryramid", and
>> we strive to do as much tests automated on a rather low level, since
>> testing
>> becomes more complicated, brittle and context dependent on higher levels of
>> integration.
>>
>> Thus I'd prefer to run those engine level tests without any connection to
>> any audio or midi system. Why? because I'd like to allow running such tests
>> in a docker container, possibly even within some cloud service (like AWS),
>> because this is an convenient and cheap way to ensure a comparable and
>> reproducible execution environment over an extended period of time.
>>
>> Of course as a standard routine, it will always be possible to run the
>> tests just on the local machine for development. It is important for
>> such tests to be very easy to start, but also flexible in the setup.
>>
>>
>> From a little bit of looking around and some experimentation I get the
>> impression this is not possible right now, since Yoshimi attempts to
>> auto-connect to ALSA. However, when I install and start yoshimi in
>> a docker container with debian, it picks the "no_audio" backend,
>> but segfaults then (don't bother, I'll find out what is going on
>> there soon...)
>>
>> Questions:
>>
>> - is there some flag or option to start yoshimi with this "no_audio" and
>> "no_midi" backend right away? Maybe we'd have to create such an option?
>>
>> - is it possible to launch yoshimi and immediately execute a CLI script?
>> Ideally yoshimi should remain in CLI mode afterwards, unless the script
>> ends with an "exit" command, in which case yoshimi should exit without
>> much further ado.
>>
>>
>> Currently I'd prefer to build some dedicated test commands into the CLI.
>> But obviously, having a dedicated commandline argument to launch a test
>> would be another option. A third (less preferrable IMHO) option would be
>> to build a dedicated test executable, which links against yoshimi code.
>> Of course, tests would always start yoshimi with a state file, in order
>> to get a controlled environment.
>>
>> As said, I am in the brainstorming phase and happy to pick up any
>> other proposed approach that is viable
>
>What about using LV2? I believe it checks off most of the boxes your
>brought up:
>
>* It uses audio and MIDI, but the input is completely deterministic from
>what you give it, and does not depend on timing. In other words, calling
>each period cycle slowly, or as fast as possible, makes no difference to
>the output.
>
>* The entrypoint API into the code is well defined, and we don't have to
>invent anything new.
>
>* The way to link it to Yoshimi would be by plugin loading. This would
>be a dedicated test binary, so not your most preferred option, but it
>should work.
>
>* Bringing up and shutting down Yoshimi is built into the LV2 loading
>procedure.
>
>* Loading a state file is also built into LV2, you simply pass in the
>state XML.
>
>* Somewhat secondary but still a bonus: We test LV2 in the process.
>
>* Downside: If you don't know LV2, it is kind of hard to understand at
>first, but it does make sense! (spoken from someone who only recently
>got into it himself).
>
>
>You would of course need to develop some of the basic LV2 loading code,
>but we would only need the very basics to get tests running, nowhere
>near a complete implementation.
>
Back from holiday :)
Another way, you could immediately get very complete access, is running scripts
from the CLI. These are plain text files such as the one below. It has a
deliberate error to demonstrate how it behaves. It occurs to me that we could
easily add a new command 'seed {n}' which would set random to a known value for
just this sort of thing.
From a terminal start yoshimi, then enter 'run {full path to script}'
# a test script
set system reverb
# a comment
# another comment
set part on
# the last part seen (1 by default)
# also makes sure it's active!
set instrument 51
# loaded from current bank
wait 20
# (mS) so it loads before next commands
s vol 30
s pan 40
s kit
# kit mode has to be set explicitly
# e.g. set crossfade
# more ok lines that won't be run
/
s p 2 on
s ins 4
l pa
--
Will J Godfrey
https://willgodfrey.bandcamp.com/
http://yoshimi.github.io
Say you have a poem and I have a tune.
Exchange them and we can both have a poem, a tune, and a song.
_______________________________________________
Yoshimi-devel mailing list
Yoshimi-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/yoshimi-devel
