Do you think the testsuite is tough enough to merge the "refactor" branch now ?
2014-05-19 20:37 GMT+02:00 Richard Shann <[email protected]>: > On Mon, 2014-05-19 at 20:11 +0200, Éloi Rivard wrote: > > What do you think of the testsuite at the moment ? I though about > > something, tell me if it looks a good idea to you. > > > > > > I create a new test that will try to execute every not builtin scheme > > command. > > > > > > The test parses the action directory to find the scheme commands. > > > > For each command, it checks if an associated test file exists, lets > > say tests/fixtures/scheme/THECOMMAND.scm. If so it executes it. > Is that to say it executes the script starting with a blank score? Does > it then save the score after the script has executed and test against > THECOMMAND.denemo ? > this would sound like a good framework for testing. > > > > If not it just executes "(d-THECOMMAND)(d-quit)". > > > > This would be a weak test in that case, > It could be made quite a bit stronger by making the environment in which > (d-THECOMMAND) is executed a more typical environment, by installing a > couple of staffs and some chords, leaving the cursor on a chord. Many > more commands do useful things when the cursor is on something and when > other staffs are present than do something useful in a completely empty > score. > > (d-AddAfter) > (d-A) > (d-CursorUp) > (d-CursorUp) > (d-AddNoteToChord) > (d-MoveCursorLeft) > (d-THECOMMAND) > (d-Save "filename= ....") > (d-Quit) > > would generate a distinctive output file for many commands (it creates > two staffs, populates one and then executes THECOMMAND in that > situation). > > > but it could at least check that the command does not provoke a > > segfault. > > > > > > > Then the test could be a bit tougher. For example, we could decide > > that if a scheme command return FALSE, it makes the test fail. > I'm not sure that Denemo commands return anything useful. But detecting > scheme exceptions would be good - we could exit in the trap handler if > noninteractive was set. We might have to fix one or two commands that > don't expect to be executed in the given environment - well, we could > just write a test for them. > > > > What do you think ? > > I think this is excellent - it will require a rule to generate the set > of expected output files, rather than diff them (for the initial > creation of the expected output), and one to accept an altered set of > files (copying them to the expected ones) would be a time-saver too. > After a change of version in the Denemo file format all the expected > output files would change - you would make just this change, check a few > examples and then run the rule to overwrite all the old versions with > the new ones. > > Richard > > > > > > > > > 2014-04-15 21:10 GMT+02:00 Richard Shann <[email protected]>: > > On Tue, 2014-04-15 at 19:13 +0100, Richard Shann wrote: > > > As you remarked, it will be good to generate a new .scm > > script each > > > time > > > a new command is made > > > > This script could assume that a variable, > > Denemo-output-filename say, > > was defined which it should use via > > > > (d-Save Denemo-output-filename) > > (d-Quit) > > > > at the end of the test. (I think I missed the (d-Quit) out of > > the > > current script ...) > > > > Richard > > > > > > > > > > > > -- > > Éloi Rivard - [email protected] > > > > « On perd plus à être indécis qu'à se tromper. » > > > > > -- Éloi Rivard - [email protected] « On perd plus à être indécis qu'à se tromper. »
_______________________________________________ Denemo-devel mailing list [email protected] https://lists.gnu.org/mailman/listinfo/denemo-devel
