On Tue, May 9, 2023 at 7:17 AM Jose E. Roman <jro...@dsic.upv.es> wrote:
> Hi. > > We are seeing a strange thing in the -log_view output with one of the > SLEPc solvers. It is probably an issue with SLEPc, but we don't know how to > debug it. > > It can be reproduced for instance with > > $ ./ex45 -m 15 -n 20 -p 21 -svd_nsv 4 -svd_ncv 9 -log_view > > The log_view events are listed at the end of this email. The first one > (PetscBarrier) is wrong, because PetscBarrier is never called, if I place a > breakpoint in PetscBarrier() it will never be hit. Also, in that event it > reports some nonzero Mflop/s, which suggests that it corresponds to another > event (not PetscBarrier). Furthermore, the count of the PetscBarrier event > always matches the count of MatMultTranspose, so there must be a connection. > > Does anyone have suggestions how to address this? > Hi Jose, Here is my guess. PETSc sets all of the event ids (using Register) when the dynamic libraries get loaded. If they are not loaded, then the library initialization function is called when some function from that library is used. My guess is that we put this init check in MatCreate(), but that is not called when you create your shell matrix and thus the events are not initialized correctly for you until later. Can you check? Thanks, Matt > Note: this is with 1 MPI process. > Note: the solver creates a shell matrix with MATOP_MULT_TRANSPOSE. > > Thanks. > Jose > > > PetscBarrier 16 1.0 7.5579e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > MatMult 82 1.0 6.1590e-01 1.0 9.25e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 97 0 0 0 81 97 0 0 0 2 > MatMultTranspose 16 1.0 7.4625e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > MatAssemblyBegin 4 1.0 5.2452e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatAssemblyEnd 4 1.0 2.8920e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatTranspose 2 1.0 1.6265e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > SVDSetUp 1 1.0 1.8686e-02 1.0 2.72e+02 1.0 0.0e+00 0.0e+00 > 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 > SVDSolve 1 1.0 5.5965e-01 1.0 7.95e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 74 83 0 0 0 74 83 0 0 0 1 > EPSSetUp 1 1.0 9.5146e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > EPSSolve 1 1.0 5.4082e-01 1.0 7.94e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 71 83 0 0 0 71 83 0 0 0 1 > STSetUp 1 1.0 4.8406e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > STComputeOperatr 1 1.0 1.2653e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > STApply 24 1.0 6.1569e-01 1.0 8.84e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > STMatSolve 24 1.0 6.1556e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > KSPSetUp 1 1.0 2.8465e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > KSPSolve 24 1.0 6.1551e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > KSPGMRESOrthog 480 1.0 4.3219e-01 1.0 3.98e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 57 42 0 0 0 57 42 0 0 0 1 > PCSetUp 1 1.0 3.8147e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > PCApply 504 1.0 2.1615e-02 1.0 1.01e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 3 1 0 0 0 3 1 0 0 0 0 > BVCopy 27 1.0 3.0560e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > BVMultVec 38 1.0 3.7060e-03 1.0 8.88e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 1 0 0 0 0 1 0 0 0 2 > BVMultInPlace 3 1.0 1.0681e-04 1.0 4.32e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 40 > BVDotVec 38 1.0 4.5121e-03 1.0 4.38e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 1 5 0 0 0 1 5 0 0 0 10 > BVOrthogonalizeV 20 1.0 1.3182e-02 1.0 5.33e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 2 6 0 0 0 2 6 0 0 0 4 > BVScale 24 1.0 4.8089e-04 1.0 4.80e+02 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 1 > BVNormVec 4 1.0 5.4097e-04 1.0 3.67e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 7 > BVNormalize 1 1.0 1.3154e-03 1.0 3.75e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 3 > BVSetRandom 1 1.0 9.1791e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > BVMatMultVec 19 1.0 4.6828e-01 1.0 6.99e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 62 73 0 0 0 62 73 0 0 0 1 > DSSolve 3 1.0 8.8906e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > DSVectors 7 1.0 2.8920e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > DSOther 12 1.0 3.8576e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > VecDot 4 1.0 8.7500e-05 1.0 1.56e+02 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 2 > VecMDot 488 1.0 6.0463e-04 1.0 1.98e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 21 0 0 0 0 21 0 0 0 327 > VecNorm 520 1.0 1.0412e-02 1.0 2.02e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 1 2 0 0 0 1 2 0 0 0 2 > VecScale 514 1.0 1.0245e-02 1.0 1.03e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 1 1 0 0 0 1 1 0 0 0 1 > VecCopy 62 1.0 1.8365e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > VecSet 64 1.0 4.2200e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > VecAXPY 32 1.0 4.8995e-04 1.0 1.44e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 3 > VecMAXPY 504 1.0 6.9070e-04 1.0 2.21e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 23 0 0 0 0 23 0 0 0 320 > VecPointwiseMult 504 1.0 3.3593e-04 1.0 1.01e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 1 0 0 0 0 1 0 0 0 30 > VecSetRandom 1 1.0 5.0068e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > VecNormalize 512 1.0 4.1106e-02 1.0 3.01e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 5 3 0 0 0 5 3 0 0 0 1 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>