Default renumbering is sequential. Since --download-parmetis requires --download-metis, I doubt Varun is using ParMETIS since the appropriate ICNTL flag is not explicitly set. If we go back to the original “issue”, I believe this is because you are doing an LU factorization with a large number of off diagonal pivots, check INFOG(12), which are handled dynamically, and thus may yield different factors.
Thanks, Pierre > On 16 Jan 2022, at 12:28 PM, Jose E. Roman <[email protected]> wrote: > > Probably someone else can give a better answer, but if I remember correctly > ParMetis relies on certain random number generator that makes it produce > different partitions for different runs. I think this was fixed in the > ParMetis that --download-parmetis installs, but if I am not wrong you are not > using that version. This would explain what you get. > > Jose > > >> El 16 ene 2022, a las 11:37, Varun Hiremath <[email protected]> >> escribió: >> >> Hi All, >> >> I am using SLEPc to compute eigenvalues and MUMPS for factorization. >> >> Please find attached: >> 1) A simple test program slepc_eps_mumps_test.cpp that reads a given PETSc >> matrix and computes the smallest eigenvalues using MUMPS for factorization >> 2) An example PETSc matrix MatA of size 581343 rows (sending in .gz format >> via Google drive link, please extract it "gunzip MatA.gz" before using). You >> should be able to reproduce this issue with any other matrix of a similar or >> bigger size. >> >> I notice that when I run the attached test program in parallel with the >> attached test matrix the MUMPS statistics printed (using the -eps_view >> option in the command line) change with every run. >> >> This is how I run the test: >> $ mpiexec -n 24 ./slepc_eps_mumps_test.o -nev 5 -f MatA -eps_view >> and for example, the output of this includes the following MUMPS stats >> ... >> PC Object: (st_) 24 MPI processes >> type: lu >> out-of-place factorization >> tolerance for zero pivot 2.22045e-14 >> matrix ordering: external >> factor fill ratio given 0., needed 0. >> Factored matrix follows: >> Mat Object: 24 MPI processes >> type: mumps >> rows=581343, cols=581343 >> package used to perform factorization: mumps >> total: nonzeros=348236349, allocated nonzeros=348236349 >> MUMPS run parameters: >> SYM (matrix type): 0 >> ... >> >> I ran this test 10 times as follows and got a different number of nonzeros >> (line highlighted above ) reported in each run. (If you save the full output >> and compare, you will notice many other differences, but I wouldn't have >> expected the nonzeros to change with every run.) >> >> $ for i in `seq 1 10`; do echo "run $i :-----"; mpiexec -n 24 >> ./slepc_eps_mumps_test.o -nev 5 -f MatA -eps_view | grep -A 1 >> "factorization: mumps"; done >> >> run 1 :----- >> package used to perform factorization: mumps >> total: nonzeros=354789915, allocated nonzeros=354789915 >> run 2 :----- >> package used to perform factorization: mumps >> total: nonzeros=359811101, allocated nonzeros=359811101 >> run 3 :----- >> package used to perform factorization: mumps >> total: nonzeros=354834871, allocated nonzeros=354834871 >> run 4 :----- >> package used to perform factorization: mumps >> total: nonzeros=354830397, allocated nonzeros=354830397 >> run 5 :----- >> package used to perform factorization: mumps >> total: nonzeros=353942929, allocated nonzeros=353942929 >> run 6 :----- >> package used to perform factorization: mumps >> total: nonzeros=354147241, allocated nonzeros=354147241 >> run 7 :----- >> package used to perform factorization: mumps >> total: nonzeros=354980083, allocated nonzeros=354980083 >> run 8 :----- >> package used to perform factorization: mumps >> total: nonzeros=354980083, allocated nonzeros=354980083 >> run 9 :----- >> package used to perform factorization: mumps >> total: nonzeros=354214219, allocated nonzeros=354214219 >> run 10 :----- >> package used to perform factorization: mumps >> total: nonzeros=355894047, allocated nonzeros=355894047 >> >> Can somebody please explain what causes these differences in MUMPS stats? >> >> Thanks, >> Varun >> MatA.gz >> >> <slepc_eps_mumps_test.cpp> >
