~~~~
Cheers,
Toby
________________________________________
From: [email protected] [[email protected]] on behalf
of Scott Kruger [[email protected]]
Sent: Tuesday, January 24, 2017 10:10
Cc: petsc-dev
Subject: Re: [petsc-dev] Introducing new "test harness" to PETSc
On 1/23/17 11:23 PM, Tobin Isaac wrote:
How does tab completion work? I've tried `make -f gmakefile sys<Tab>`
in zsh and bash but got nothing.
Good question. I think this is handled by the bash-completion project
which is normally an apt-get install or brew install, but has never
worked on my mac for some reason.
Thanks to everyone who worked on this, I just have two other
questions/requests:
- What’s the best way to update the test definitions after fiddling
with /*TEST*/ blocks? It would be nice if gmakegentests.py would run
automatically when sources are newer than their tests.
I'll let Jed comment on this. It's not trivial with the current method
of specifying targets (gmake's lisp-like language), and even my
first pedantic implementation (explicitly writing out targets for
each test), I seem to recall it being brittle because of the global
nature of the gmakegentest.py requiring the config.
My development workflow is generally:
- edit source file
- run gmakegentest.py
- inspect shell script in $PETSC_ARCH/tests
- run shell script by hand and edit as needed
- test with makefile (either search or global) before commit
There is also config/testparse.py which can be run on an
individual file if you just want to test the parsing:
> $PETSC_DIR/config/testparse.py -v 1 -t ex1.c
ex1.c
runex1
filter: egrep "(PETSC ERROR)" | egrep "(main|CreateError|Error
Created)" | cut -f1,2,3,4,5,6 -d" "
- Is there a technical reason suffixes can only be specified at the
top level? It would help with concision if subtest suffixes could
append to their parent suffixes.
Design decision on what should be a test an what should be a subtest.
If you want your test to go to a different file, then it is now a test.
Trying to do something more complicated would add
unnecessary complexity. If you do want some grouping, then it can
be done with suffix itself; i.e., suffix does not have to be an
integer, but can be an arbitrary string.
But perhaps I don't understand your use case.
Scott
Cheers,
Toby
*From:* Jed Brown <mailto:[email protected]>
*Sent:* Tuesday, 17 January, 2017 20:55
*To:* Matthew Knepley <mailto:[email protected]>, Barry Smith
<mailto:[email protected]>
*Cc:* petsc-dev <mailto:[email protected]>
Matthew Knepley <[email protected]> writes:
Gripes:
1) How do I run an individual test run?
make -f gmakefile sys_tutorials-runex5
(tab completion works) or use a search that matches only that run.
I use
make -f gmakefile test searchin=plex
to run plex tests, and I think
make -f gmakefile test searchin=plex*ex1
Due to make syntax, % is the internal string matcher.
will work. However, how do I select ex1_2?
make -f gmakefile test search=plex%ex1_2
2) How do I replace the output which exists with the output of the
current test run?
This is amazingly useful after changes.
I have this script ~/bin/diffupdate:
#!/bin/sh
args=
if [ "$1" = "-u" ]; then
args="-u"
shift
fi
new="$2"
ref="$1"
diff $args "$ref" "$new"
ret=$?
echo "Updating $new --> $ref"
mv "$new" "$ref"
exit $ret
Then I run
make -f gmakefile test search=plex% DIFF=diffupdate
It prints the diff and replaces the reference file (first argument) with
the second.
3) How do I give extra arguments with the make interface, rather than
test_harness?
That is, run some modification of the test and still do the diff, but
don't change the reference output? I have used PETSC_OPTIONS for that.
make .... PETSC_OPTIONS=-more_magic
Once I have all the functionality of my Python stuff, I will throw it
away.
--
Tech-X Corporation [email protected]
5621 Arapahoe Ave, Suite A Phone: (720) 974-1841
Boulder, CO 80303 Fax: (303) 448-7756
