Re: [petsc-users] MPI linear solver reproducibility question

2023-04-02 Thread Mark McClure
Ok, good to know. I'll update to latest Petsc, and do some testing, and let you know either way. On Sun, Apr 2, 2023 at 6:31 AM Jed Brown wrote: > Vector communication used a different code path in 3.13. If you have a > reproducer with current PETSc, I'll have a look. Here's a demo that the >

Re: [petsc-users] MPI linear solver reproducibility question

2023-04-02 Thread Jed Brown
Vector communication used a different code path in 3.13. If you have a reproducer with current PETSc, I'll have a look. Here's a demo that the solution is bitwise identical (the sha256sum is the same every time you run it, though it might be different on your computer from mine due to compiler

Re: [petsc-users] MPI linear solver reproducibility question

2023-04-01 Thread Mark McClure
In the typical FD implementation, you only set local rows, but with FE and sometimes FV, you also create values that need to be communicated and summed on other processors. Makes sense. Anyway, in this case, I am certain that I am giving the solver bitwise identical matrices from each process. I

Re: [petsc-users] MPI linear solver reproducibility question

2023-04-01 Thread Jed Brown
Mark McClure writes: > Thank you, I will try BCGSL. > > And good to know that this is worth pursuing, and that it is possible. Step > 1, I guess I should upgrade to the latest release on Petsc. > > How can I make sure that I am "using an MPI that follows the suggestion for > implementers about

Re: [petsc-users] MPI linear solver reproducibility question

2023-04-01 Thread Mark McClure
Thank you, I will try BCGSL. And good to know that this is worth pursuing, and that it is possible. Step 1, I guess I should upgrade to the latest release on Petsc. How can I make sure that I am "using an MPI that follows the suggestion for implementers about determinism"? I am using MPICH

Re: [petsc-users] MPI linear solver reproducibility question

2023-04-01 Thread Jed Brown
If you use unpreconditioned BCGS and ensure that you assemble the same matrix (depends how you do the communication for that), I think you'll get bitwise reproducible results when using an MPI that follows the suggestion for implementers about determinism. Beyond that, it'll depend somewhat on

[petsc-users] MPI linear solver reproducibility question

2023-04-01 Thread Mark McClure
Hello, I have been a user of Petsc for quite a few years, though I haven't updated my version in a few years, so it's possible that my comments below could be 'out of date'. Several years ago, I'd asked you guys about reproducibility. I observed that if I gave an identical matrix to the Petsc