I still don't understand. Can you point to a test and a subtest among the directories that have been converted so far. Or have no subtests been converted yet?
> On Jan 24, 2017, at 2:56 PM, Scott Kruger <[email protected]> wrote: > > > > On 1/24/17 9:57 AM, Barry Smith wrote: >> >> I don't really understand what a "sub-test" is, just any test in an example >> that is not first one? Aside from the suffix are "sub-tests" handled any >> different than the first test? I don't think they should be. > > subtest has a one-to-one correspondence to an mpiexec call > A test has a one-to-one correspondence to a shell script and make target. > > For-loops that invoke mpiexec call multiple times => multiple sub-tests. > There are also just multiple invocations by hand. > > How to handle diffs? > > Ultimately, it was decided to have one output file per shell script > to avoid making things too confusing. > > convertExamples.py handles most of these "reasonably" well. > > Scott > > >> >>> On Jan 24, 2017, at 10:10 AM, Scott Kruger <[email protected]> wrote: >>> >>> >>> >>> On 1/23/17 11:23 PM, Tobin Isaac wrote: >>>> How does tab completion work? I've tried `make -f gmakefile sys<Tab>` >>>> in zsh and bash but got nothing. >>> >>> Good question. I think this is handled by the bash-completion project >>> which is normally an apt-get install or brew install, but has never >>> worked on my mac for some reason. >>> >>>> >>>> Thanks to everyone who worked on this, I just have two other >>>> questions/requests: >>>> >>>> - What’s the best way to update the test definitions after fiddling >>>> with /*TEST*/ blocks? It would be nice if gmakegentests.py would run >>>> automatically when sources are newer than their tests. >>> >>> I'll let Jed comment on this. It's not trivial with the current method >>> of specifying targets (gmake's lisp-like language), and even my >>> first pedantic implementation (explicitly writing out targets for >>> each test), I seem to recall it being brittle because of the global >>> nature of the gmakegentest.py requiring the config. >>> >>> My development workflow is generally: >>> - edit source file >>> - run gmakegentest.py >>> - inspect shell script in $PETSC_ARCH/tests >>> - run shell script by hand and edit as needed >>> - test with makefile (either search or global) before commit >>> >>> There is also config/testparse.py which can be run on an >>> individual file if you just want to test the parsing: >>> >>>> $PETSC_DIR/config/testparse.py -v 1 -t ex1.c >>> ex1.c >>> runex1 >>> filter: egrep "(PETSC ERROR)" | egrep "(main|CreateError|Error >>> Created)" | cut -f1,2,3,4,5,6 -d" " >>> >>>> >>>> - Is there a technical reason suffixes can only be specified at the >>>> top level? It would help with concision if subtest suffixes could >>>> append to their parent suffixes. >>> >>> Design decision on what should be a test an what should be a subtest. >>> If you want your test to go to a different file, then it is now a test. >>> Trying to do something more complicated would add >>> unnecessary complexity. If you do want some grouping, then it can >>> be done with suffix itself; i.e., suffix does not have to be an >>> integer, but can be an arbitrary string. >>> >>> But perhaps I don't understand your use case. >>> >>> Scott >>> >>> >>> >>>> >>>> Cheers, >>>> Toby >>>> >>>> *From:* Jed Brown <mailto:[email protected]> >>>> *Sent:* Tuesday, 17 January, 2017 20:55 >>>> *To:* Matthew Knepley <mailto:[email protected]>, Barry Smith >>>> <mailto:[email protected]> >>>> *Cc:* petsc-dev <mailto:[email protected]> >>>> >>>> Matthew Knepley <[email protected]> writes: >>>>> Gripes: >>>>> >>>>> 1) How do I run an individual test run? >>>> >>>> make -f gmakefile sys_tutorials-runex5 >>>> >>>> (tab completion works) or use a search that matches only that run. >>>> >>>>> I use >>>>> >>>>> make -f gmakefile test searchin=plex >>>>> >>>>> to run plex tests, and I think >>>>> >>>>> make -f gmakefile test searchin=plex*ex1 >>>> >>>> Due to make syntax, % is the internal string matcher. >>>> >>>>> will work. However, how do I select ex1_2? >>>> >>>> make -f gmakefile test search=plex%ex1_2 >>>> >>>>> 2) How do I replace the output which exists with the output of the >>>>> current test run? >>>>> >>>>> This is amazingly useful after changes. >>>> >>>> I have this script ~/bin/diffupdate: >>>> #!/bin/sh >>>> >>>> args= >>>> if [ "$1" = "-u" ]; then >>>> args="-u" >>>> shift >>>> fi >>>> new="$2" >>>> ref="$1" >>>> >>>> diff $args "$ref" "$new" >>>> ret=$? >>>> echo "Updating $new --> $ref" >>>> mv "$new" "$ref" >>>> exit $ret >>>> >>>> >>>> Then I run >>>> >>>> make -f gmakefile test search=plex% DIFF=diffupdate >>>> >>>> It prints the diff and replaces the reference file (first argument) with >>>> the second. >>>> >>>>> 3) How do I give extra arguments with the make interface, rather than >>>>> test_harness? >>>> >>>> That is, run some modification of the test and still do the diff, but >>>> don't change the reference output? I have used PETSC_OPTIONS for that. >>>> >>>> make .... PETSC_OPTIONS=-more_magic >>>> >>>>> Once I have all the functionality of my Python stuff, I will throw it >>>> away. >>>> >>> >>> -- >>> Tech-X Corporation [email protected] >>> 5621 Arapahoe Ave, Suite A Phone: (720) 974-1841 >>> Boulder, CO 80303 Fax: (303) 448-7756 >> > > -- > Tech-X Corporation [email protected] > 5621 Arapahoe Ave, Suite A Phone: (720) 974-1841 > Boulder, CO 80303 Fax: (303) 448-7756
