Most of them are in mat and ksp/ksp.

See ksp/ksp/examples/tests:runex32_testset1 for an extreme for loop example.

See mat/examples/tests:runex2 for multiple subtests within a single target.

There are others in those makefiles as well (and tutorials dirs).

Needless to say, adding subtest functionality was somewhat challenging,
and automating the conversion even more so.  I hope folks find it useful!

Scott


On 1/24/17 2:00 PM, Barry Smith wrote:

  I still don't understand. Can you point to a test and a subtest among the 
directories that have been converted so far. Or have no subtests been converted 
yet?


On Jan 24, 2017, at 2:56 PM, Scott Kruger <[email protected]> wrote:



On 1/24/17 9:57 AM, Barry Smith wrote:

 I don't really understand what a "sub-test" is, just any test in an example that is not 
first one? Aside from the suffix are "sub-tests" handled any different than the first 
test? I don't think they should be.

subtest has a one-to-one correspondence to an mpiexec call
A test has a one-to-one correspondence to a shell script and make target.

For-loops that invoke mpiexec call multiple times => multiple sub-tests.
There are also just multiple invocations by hand.

How to handle diffs?

Ultimately, it was decided to have one output file per shell script
to avoid making things too confusing.

convertExamples.py handles most of these "reasonably" well.

Scott



On Jan 24, 2017, at 10:10 AM, Scott Kruger <[email protected]> wrote:

        

On 1/23/17 11:23 PM, Tobin Isaac wrote:
How does tab completion work?  I've tried `make -f gmakefile sys<Tab>`
in zsh and bash but got nothing.

Good question.   I think this is handled by the bash-completion project
which is normally an apt-get install or brew install, but has never
worked on my mac for some reason.


Thanks to everyone who worked on this, I just have two other
questions/requests:

- What’s the best way to update the test definitions after fiddling
with /*TEST*/ blocks?  It would be nice if gmakegentests.py would run
automatically when sources are newer than their tests.

I'll let Jed comment on this.  It's not trivial with the current method
of specifying targets (gmake's lisp-like language), and even my
first pedantic implementation (explicitly writing out targets for
each test), I seem to recall it being brittle because of the global
nature of the gmakegentest.py requiring the config.

My development workflow is generally:
- edit source file
- run gmakegentest.py
- inspect shell script in $PETSC_ARCH/tests
- run shell script by hand and edit as needed
- test with makefile (either search or global) before commit

There is also config/testparse.py which can be run on an
individual file if you just want to test the parsing:

$PETSC_DIR/config/testparse.py -v 1 -t ex1.c
ex1.c
 runex1
    filter: egrep "(PETSC ERROR)" | egrep "(main|CreateError|Error Created)" | cut 
-f1,2,3,4,5,6 -d" "


- Is there a technical reason suffixes can only be specified at the
top level?  It would help with concision if subtest suffixes could
append to their parent suffixes.

Design decision on what should be a test an what should be a subtest.
If you want your test to go to a different file, then it is now a test.
Trying to do something more complicated would add
unnecessary complexity.  If you do want some grouping, then it can
be done with suffix itself; i.e., suffix does not have to be an
integer, but can be an arbitrary string.

But perhaps I don't understand your use case.

Scott




Cheers,
Toby

*From:* Jed Brown <mailto:[email protected]>
*Sent:* ‎Tuesday‎, ‎17‎ ‎January‎, ‎2017 ‎20‎:‎55
*To:* Matthew Knepley <mailto:[email protected]>, Barry Smith
<mailto:[email protected]>
*Cc:* petsc-dev <mailto:[email protected]>

Matthew Knepley <[email protected]> writes:
Gripes:

1) How do I run an individual test run?

make -f gmakefile sys_tutorials-runex5

(tab completion works) or use a search that matches only that run.

I use

make -f gmakefile test searchin=plex

to run plex tests, and I think

make -f gmakefile test searchin=plex*ex1

Due to make syntax, % is the internal string matcher.

will work. However, how do I select ex1_2?

make -f gmakefile test search=plex%ex1_2

2) How do I replace the output which exists with the output of the
current test run?

This is amazingly useful after changes.

I have this script ~/bin/diffupdate:
#!/bin/sh

args=
if [ "$1" = "-u" ]; then
  args="-u"
  shift
fi
new="$2"
ref="$1"

diff $args "$ref" "$new"
ret=$?
echo "Updating $new --> $ref"
mv "$new" "$ref"
exit $ret


Then I run

make -f gmakefile test search=plex% DIFF=diffupdate

It prints the diff and replaces the reference file (first argument) with
the second.

3) How do I give extra arguments with the make interface, rather than
test_harness?

That is, run some modification of the test and still do the diff, but
don't change the reference output?  I have used PETSC_OPTIONS for that.

make .... PETSC_OPTIONS=-more_magic

Once I have all the functionality of my Python stuff,  I will throw it
away.


--
Tech-X Corporation               [email protected]
5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
Boulder, CO 80303                Fax:   (303) 448-7756


--
Tech-X Corporation               [email protected]
5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
Boulder, CO 80303                Fax:   (303) 448-7756


--
Tech-X Corporation               [email protected]
5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
Boulder, CO 80303                Fax:   (303) 448-7756

Reply via email to