On Tue, Jan 17, 2017 at 4:19 PM, Matthew Knepley <[email protected]> wrote:
> On Fri, Jan 13, 2017 at 4:16 PM, Barry Smith <[email protected]> wrote: > >> >> PETSc developers, >> >> Scott Kruger has been implementing a new "test harness" for PETSc. >> Rather than having tests implemented as bash lines in the makefiles, the >> tests will be defined within the source code for each example with a simple >> format and the actual scripts that run the tests will be automatically >> generated. This offers three advantages over the current system >> >> 1) test definitions can be given in a very concise format that is >> independent of how the test is run >> 2) multiple approaches to actually running the tests can be implemented >> without requiring any changes to the test definitions for each example. >> 3) the tests can be compiled and run in parallel (eventually we hope to >> see much faster times for testing). >> >> Here is an example of the format that defines tests >> >> test: >> >> test: >> suffix: yaml >> requires: yaml !complex >> args: -options_file_yaml bag.yml -options_view >> filter: grep -v saws_port_auto_select >> localrunfiles: bag.yml >> > > How do I specify the number of processes to use for the test? Or where > would I look this up? > Also, how do I put comments inside these blocks to tell me what the test is about? Matt > Thanks, > > Matt > > >> The first is a test that takes no command line arguments and does no >> filtering of output. The second takes command line arguments, does some >> filtering of the output and this example also requires a data file >> associated with that example. The expected output of tests continues to >> reside in the output directory with the previously naming convention of >> XXX_1.out XXX_2.out where XXX is the name of the example or XXX_suffix.out >> where suffix is defined in the test definition. >> >> So far in the master branch we have only converted over two directories >> of examples sys/examples/tutorials and sys/examples/tests so you can look >> in those directories for other examples. Over the next few weeks we hope to >> convert over all the rest of the directories. >> >> Special thanks to Scott, Satish, and Jed for all the work they have >> done to set up the new test harness. >> >> Barry >> >> Technical details: the new python script ./config/gmakegentest.py >> which is normally automatically run at configure walks through the examples >> directories looking for the formatted test definitions, for each test it >> generates in the directory $PETSC_ARCH/test/src/XXX/examples/tests or >> tutorials/ a shell script that can run that test. In addition >> config/gmakegentest.py adds dependencies to the $PETSC_DIR/gmakefile for >> all the tests that it has found. The rule make alltests then executes all >> those dependencies. >> >> We will be refining both the testing harness and the test definitions >> as we convert the rest of the examples over to the new approach. >> >> >> >> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
