hm - indices are integers and its not ignored (i.e %d) - only the %f and %e diff is ignored (by default)
Satish On Mon, 4 Jan 2021, Scott Kruger wrote: > > > > Is this just the 3rd problem? > > Regarding how you can end up with changes not being caught: > The default (going all the way back to the old harness) is to not check > numbers to avoid round-off errors giving false negatives (failures). > Of course, sometimes you *want* to check the numbers; e.g., for indices. The > solution for this is to add: > > diff_args: -j > > Scott > > On 12/31/20 4:40 PM, Barry Smith wrote: > > > > I think I have it "fixed" now in the branch, once it passes the pipeline I > > will shepard it through the MR quickly. Sorry about this, even all our CI > > testing can miss a great deal. > > > > Barry > > > > > > > >> On Dec 31, 2020, at 2:44 PM, Barry Smith <[email protected] > >> <mailto:[email protected]>> wrote: > >> > >> > >> This is a different (3rd) problem. Funny it didn't bother anyone for two > >> months. > >> > >> Fix is in barry/2020-12-29/fix-petscdiff-bracket but the pipeline keeps > >> failing ts_tutorials_advection-diffusion-reaction-ex3_2 fails on different > >> machines with slightly different counts. I don't see how this change could > >> cause that! But gets old results on my machine. Very frustrating. > >> Barry > >> > >>> On Dec 31, 2020, at 1:02 PM, Matthew Knepley <[email protected] > >>> <mailto:[email protected]>> wrote: > >>> > >>> On Thu, Dec 31, 2020 at 1:48 PM Barry Smith <[email protected] > >>> <mailto:[email protected]>> wrote: > >>> > >>> > >>> So the programs output changes and should no longer match that > >>> in the output/* file yet the test harness does not error with a > >>> statement that the two outputs do not match? > >>> > >>> I noticed the gmakegentest.py is not being run before it runs > >>> the test? Does this mean it is just running all the old stuff > >>> which does match fine? > >>> > >>> Then either how petscdiff is called by the test harness has > >>> changed or petscdiff has changed and does not detect changes > >>> anymore > >>> > >>> BTW: I always use -f ./gmakefile.test test not just the gmakefile > >>> > >>> All the PETSc changes are trivial and can be seen with a > >>> simple diff, it is hard to believe they would cause this > >>> behavior but I guess they must. > >>> > >>> You can go to PETSC_ARCH/tests/snes/tests and run the ex13 > >>> shell script directly. > >>> > >>> > >>> It is the sed problem: > >>> > >>> master *$:/PETSc3/petsc/petsc-dev$ > >>> /PETSc3/petsc/petsc-dev/lib/petsc/bin/petscdiff > >>> /PETSc3/petsc/petsc-dev/src/snes/tests/output/ex13_bench.out > >>> ex13_bench.tmp > >>> > >>> sed: 1: "s/\033[1;31m//g": unbalanced brackets ([]) > >>> sed: 1: "s/\033[0;39m\033[0;49m//g": unbalanced brackets ([]) > >>> sed: 1: "s/\033[1;31m//g": unbalanced brackets ([]) > >>> sed: 1: "s/\033[0;39m\033[0;49m//g": unbalanced brackets ([]) > >>> > >>> The error was getting eaten. > >>> > >>> This is in current master. Is it fixed in a branch? > >>> > >>> Matt > >>> > >>> Barry > >>> > >>> > >>>> On Dec 31, 2020, at 12:38 PM, Matthew Knepley > >>>> <[email protected] <mailto:[email protected]>> wrote: > >>>> > >>>> I just pulled master, and simple alterations to tests do not > >>>> produce a failure: > >>>> > >>>> master *$:/PETSc3/petsc/petsc-dev$ PETSC_ARCH=arch-master-debug > >>>> make -f ./gmakefile test search="snes_tests-ex13_bench" > >>>> TIMEOUT=5000 EXTRA_OPTIONS="-dm_ > >>>> refine 0" > >>>> Using MAKEFLAGS: EXTRA_OPTIONS=-dm_refine 0 TIMEOUT=5000 > >>>> search=snes_tests-ex13_bench > >>>> TEST > >>>> arch-master-debug/tests/counts/snes_tests-ex13_bench.counts > >>>> ok snes_tests-ex13_bench > >>>> ok diff-snes_tests-ex13_bench > >>>> > >>>> I check that the runs produce different output when done manually. > >>>> > >>>> Scott and Barry, could this be related to changed to testing? > >>>> > >>>> Thanks, > >>>> > >>>> Matt > >>>> > >>>> -- > >>>> What most experimenters take for granted before they begin > >>>> their experiments is infinitely more interesting than any > >>>> results to which their experiments lead. > >>>> -- Norbert Wiener > >>>> > >>>> https://www.cse.buffalo.edu/~knepley/ > >>>> <http://www.cse.buffalo.edu/%7Eknepley/> > >>> > >>> > >>> > >>> -- > >>> What most experimenters take for granted before they begin their > >>> experiments is infinitely more interesting than any results to which their > >>> experiments lead. > >>> -- Norbert Wiener > >>> > >>> https://www.cse.buffalo.edu/~knepley/ > >>> <http://www.cse.buffalo.edu/%7Eknepley/> > >> > > > >
