Is this just the 3rd problem?
Regarding how you can end up with changes not being caught:
The default (going all the way back to the old harness) is to not check
numbers to avoid round-off errors giving false negatives (failures).
Of course, sometimes you *want* to check the numbers; e.g., for
indices. The solution for this is to add:
diff_args: -j
Scott
On 12/31/20 4:40 PM, Barry Smith wrote:
I think I have it "fixed" now in the branch, once it passes the
pipeline I will shepard it through the MR quickly. Sorry about this,
even all our CI testing can miss a great deal.
Barry
On Dec 31, 2020, at 2:44 PM, Barry Smith <[email protected]
<mailto:[email protected]>> wrote:
This is a different (3rd) problem. Funny it didn't bother anyone
for two months.
Fix is in barry/2020-12-29/fix-petscdiff-bracket but the pipeline
keeps failing ts_tutorials_advection-diffusion-reaction-ex3_2 fails
on different machines with slightly different counts. I don't see how
this change could cause that! But gets old results on my machine.
Very frustrating.
Barry
On Dec 31, 2020, at 1:02 PM, Matthew Knepley <[email protected]
<mailto:[email protected]>> wrote:
On Thu, Dec 31, 2020 at 1:48 PM Barry Smith <[email protected]
<mailto:[email protected]>> wrote:
So the programs output changes and should no longer match that
in the output/* file yet the test harness does not error with a
statement that the two outputs do not match?
I noticed the gmakegentest.py is not being run before it runs
the test? Does this mean it is just running all the old stuff
which does match fine?
Then either how petscdiff is called by the test harness has
changed or petscdiff has changed and does not detect changes
anymore
BTW: I always use -f ./gmakefile.test test not just the gmakefile
All the PETSc changes are trivial and can be seen with a
simple diff, it is hard to believe they would cause this
behavior but I guess they must.
You can go to PETSC_ARCH/tests/snes/tests and run the ex13
shell script directly.
It is the sed problem:
master *$:/PETSc3/petsc/petsc-dev$
/PETSc3/petsc/petsc-dev/lib/petsc/bin/petscdiff
/PETSc3/petsc/petsc-dev/src/snes/tests/output/ex13_bench.out
ex13_bench.tmp
sed: 1: "s/\033[1;31m//g": unbalanced brackets ([])
sed: 1: "s/\033[0;39m\033[0;49m//g": unbalanced brackets ([])
sed: 1: "s/\033[1;31m//g": unbalanced brackets ([])
sed: 1: "s/\033[0;39m\033[0;49m//g": unbalanced brackets ([])
The error was getting eaten.
This is in current master. Is it fixed in a branch?
Matt
Barry
On Dec 31, 2020, at 12:38 PM, Matthew Knepley
<[email protected] <mailto:[email protected]>> wrote:
I just pulled master, and simple alterations to tests do not
produce a failure:
master *$:/PETSc3/petsc/petsc-dev$ PETSC_ARCH=arch-master-debug
make -f ./gmakefile test search="snes_tests-ex13_bench"
TIMEOUT=5000 EXTRA_OPTIONS="-dm_
refine 0"
Using MAKEFLAGS: EXTRA_OPTIONS=-dm_refine 0 TIMEOUT=5000
search=snes_tests-ex13_bench
TEST
arch-master-debug/tests/counts/snes_tests-ex13_bench.counts
ok snes_tests-ex13_bench
ok diff-snes_tests-ex13_bench
I check that the runs produce different output when done manually.
Scott and Barry, could this be related to changed to testing?
Thanks,
Matt
--
What most experimenters take for granted before they begin
their experiments is infinitely more interesting than any
results to which their experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/
<http://www.cse.buffalo.edu/%7Eknepley/>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which
their experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/
<http://www.cse.buffalo.edu/%7Eknepley/>
--
Tech-X Corporation [email protected]
5621 Arapahoe Ave, Suite A Phone: (720) 466-3196
Boulder, CO 80303 Fax: (303) 448-7756
