I don't really understand what a "sub-test" is, just any test in an example that is not first one? Aside from the suffix are "sub-tests" handled any different than the first test? I don't think they should be.
> On Jan 24, 2017, at 10:10 AM, Scott Kruger <[email protected]> wrote: > > > > On 1/23/17 11:23 PM, Tobin Isaac wrote: >> How does tab completion work? I've tried `make -f gmakefile sys<Tab>` >> in zsh and bash but got nothing. > > Good question. I think this is handled by the bash-completion project > which is normally an apt-get install or brew install, but has never > worked on my mac for some reason. > >> >> Thanks to everyone who worked on this, I just have two other >> questions/requests: >> >> - What’s the best way to update the test definitions after fiddling >> with /*TEST*/ blocks? It would be nice if gmakegentests.py would run >> automatically when sources are newer than their tests. > > I'll let Jed comment on this. It's not trivial with the current method > of specifying targets (gmake's lisp-like language), and even my > first pedantic implementation (explicitly writing out targets for > each test), I seem to recall it being brittle because of the global > nature of the gmakegentest.py requiring the config. > > My development workflow is generally: > - edit source file > - run gmakegentest.py > - inspect shell script in $PETSC_ARCH/tests > - run shell script by hand and edit as needed > - test with makefile (either search or global) before commit > > There is also config/testparse.py which can be run on an > individual file if you just want to test the parsing: > > > $PETSC_DIR/config/testparse.py -v 1 -t ex1.c > ex1.c > runex1 > filter: egrep "(PETSC ERROR)" | egrep "(main|CreateError|Error Created)" > | cut -f1,2,3,4,5,6 -d" " > >> >> - Is there a technical reason suffixes can only be specified at the >> top level? It would help with concision if subtest suffixes could >> append to their parent suffixes. > > Design decision on what should be a test an what should be a subtest. > If you want your test to go to a different file, then it is now a test. > Trying to do something more complicated would add > unnecessary complexity. If you do want some grouping, then it can > be done with suffix itself; i.e., suffix does not have to be an > integer, but can be an arbitrary string. > > But perhaps I don't understand your use case. > > Scott > > > >> >> Cheers, >> Toby >> >> *From:* Jed Brown <mailto:[email protected]> >> *Sent:* Tuesday, 17 January, 2017 20:55 >> *To:* Matthew Knepley <mailto:[email protected]>, Barry Smith >> <mailto:[email protected]> >> *Cc:* petsc-dev <mailto:[email protected]> >> >> Matthew Knepley <[email protected]> writes: >>> Gripes: >>> >>> 1) How do I run an individual test run? >> >> make -f gmakefile sys_tutorials-runex5 >> >> (tab completion works) or use a search that matches only that run. >> >>> I use >>> >>> make -f gmakefile test searchin=plex >>> >>> to run plex tests, and I think >>> >>> make -f gmakefile test searchin=plex*ex1 >> >> Due to make syntax, % is the internal string matcher. >> >>> will work. However, how do I select ex1_2? >> >> make -f gmakefile test search=plex%ex1_2 >> >>> 2) How do I replace the output which exists with the output of the >>> current test run? >>> >>> This is amazingly useful after changes. >> >> I have this script ~/bin/diffupdate: >> #!/bin/sh >> >> args= >> if [ "$1" = "-u" ]; then >> args="-u" >> shift >> fi >> new="$2" >> ref="$1" >> >> diff $args "$ref" "$new" >> ret=$? >> echo "Updating $new --> $ref" >> mv "$new" "$ref" >> exit $ret >> >> >> Then I run >> >> make -f gmakefile test search=plex% DIFF=diffupdate >> >> It prints the diff and replaces the reference file (first argument) with >> the second. >> >>> 3) How do I give extra arguments with the make interface, rather than >>> test_harness? >> >> That is, run some modification of the test and still do the diff, but >> don't change the reference output? I have used PETSC_OPTIONS for that. >> >> make .... PETSC_OPTIONS=-more_magic >> >>> Once I have all the functionality of my Python stuff, I will throw it >> away. >> > > -- > Tech-X Corporation [email protected] > 5621 Arapahoe Ave, Suite A Phone: (720) 974-1841 > Boulder, CO 80303 Fax: (303) 448-7756
