On Thu, 26 Mar 2009, Tim Kroeger wrote:
> You might want to check whether you can reproduce the non-reproducibility. > To do this, please download www.mevis.de/~tim/a.tar.gz and unpack it (small > this time). Then run the attached test.cpp on 8 CPUs, which creates a simple > grid, assembles a matrix as given by the files, writes the complete matrix to > another file (in PETSc binary format), and then solves. On solving the > system, it will crash, but that's not important in this case. (The grid is so > coarse that important geometric structures are not "seen", so that the system > becomes singular.) Run that program several times and rename the resulting > matrix files. Using "diff" you will find that they differ. You can use the > attached test2.cpp to get an ASCII representation of the matrices. I needed "#include <stdlib.h>" to get test2 to work; g++ is getting more and more nitpicky about standards compliance. I can reproduce the non-reproducibility... but not to any level that I'd worry about. The global matrices differ, but every difference was literally in the 16th or 17th decimal significant figure! I'd assume that PETSc assembles matrices in the order they're received, the order in which they're received depends on which processor happened to go slightly faster than which other processor, and so the differences we're seeing might all stem from the tragedy that, in floating point math, you can have (A+B)+C != A+(B+C). I can't say I'm happy about not having perfectly deterministic apps, but I'm not sure what to do about it. Maybe someone familiar with PETSc MatSetValues can chime in with "We're deterministic! Roy doesn't know what he's talking about" or "We *can be deterministic*, if you just configure or run with --some-obscure-option"; otherwise I'm stuck. Alternatively, maybe I *haven't* reproduced what you're seeing. IIRC you were talking about residual differences in the 6th place, not the 16th, so maybe there's something more at work there. --- Roy ------------------------------------------------------------------------------ _______________________________________________ Libmesh-devel mailing list Libmesh-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libmesh-devel
