On 1/25/18 4:35 AM, Patrick Sanan wrote:
This following is what I would try, given my limited knowledge of how
things work and after a quick scan of the docs. Is this as intended or
is there an easier way?
You pretty much have it.
--------------------------------------------------
I make some seemingly-innocuous changes to the code. (In this case I
literally change nothing in master.)
I want to run all the tests to make sure I didn't break something that I
don't understand, so I look at the User Manual and copy and run this
command:
make -f gmakefile.test test
This will be hooked up to the main makefile test target at some point.
Right now,
make allgtest
is more silent for example, but you can pick out the errors.
make allgtest-tap
is equivalent to above (you see progress, but you have to pick out
the errrors that occurred in 0.1% of tests).
I go away for a while and come back and see that some (3) tests have
failed, and that the harness gives me instructions on how to re-run them.
# -------------
# Summary
# -------------
# FAILED ts_tutorials-ex11_adv_2d_quad_0
diff-sys_classes_viewer_tests-ex4_4 ts_tutorials-ex11_adv_2d_quad_1
# success 3051/3915 tests (77.9%)
# failed 3/3915 tests (0.1%)
# todo 91/3915 tests (2.3%)
# skip 770/3915 tests (19.7%)
#
# To rerun failed tests:
# /opt/local/bin/gmake -f gmakefile test
search='ts_tutorials-runex11_adv_2d_quad_0
sys_classes_viewer_tests-runex4_4 ts_tutorials-runex11_adv_2d_quad'
I want to take a look at the output so I can see what's happening - I
see that I'm given instructions on how to get more verbose output, so I
run one of the tests again with V=1 :
/opt/local/bin/gmake V=1 -f gmakefile test
search='ts_tutorials-runex11_adv_2d_quad_0'
This tells me that this is failing because of a timeout
#Exceeded timeout limit of 60 s
I poke around in the dev manual (note that "make -f gmakefile help" and
"make -f gmakefile.test help" say nothing about TIMEOUT as promised)
Oops. Yes, a bug.
If I look in the runex4_4 directory, I find
sys_classes_viewer_tests-ex4_4.sh which gives me something to copy and
paste and use my actual knowledge/reasoning to fix:
/Users/patrick/petsc-master/arch-darwin-master-double-debug/bin/mpiexec -n 1 ../ex4 -myviewer ascii:ex4a1.tmp::append
(I'm not going to try to debug this failure further, but I'm confident
I'd be able to given the information I have now)
This is pretty close to what I want, which is a way to "back out of the
test harness" to be able to debug the failed test myself. I was hoping
that running this from the PETSc root directory would more directly tell
me that:
arch-darwin-master-double-debug/tests/src/sys/classes/viewer/examples/tests/runex4_4.sh -v
The output is meant to be user facing (in which case they
do not want to be told how to debug -- they just want to
report it), and to be run for nightlies -- in which case
we want quite a bit of terseness.
I think the fact that you figured it out on your own
is a success! You did the workflow I use and yes, the
first time is annoying, but then it's pretty straightforward.
A few comments:
The additional step of poking around the runex4_4 directory
is to avoid output file name clashes.
The runex4_4.sh script has a -h command to show other options,
especially the ability to automatically update the
output files.
NO_RM=1 is my favorite option to gmakefile.test, but
I have shared builds. Somewhere we have a change that turns this on
if shared builds are there so debugging is faster.
Scott
2018-01-25 5:47 GMT+01:00 Smith, Barry F. <[email protected]
<mailto:[email protected]>>:
PETSc developers,
We have completed moving all PETSc examples over from the old
test system (where tests were written in the makefile) to a new
system, provided by Scott Kruger, where the test rules are written
in bottom of the source file of the example. Directions for usage
and adding new tests can be found in the PETSc developers manual
http://www.mcs.anl.gov/petsc/petsc-dev/docs/developers.pdf
<http://www.mcs.anl.gov/petsc/petsc-dev/docs/developers.pdf> chapter 7.
Barry
--
Tech-X Corporation [email protected]
5621 Arapahoe Ave, Suite A Phone: (720) 974-1841
Boulder, CO 80303 Fax: (303) 448-7756
