Re: [petsc-users] memory corruption when using harmonic extraction with SLEPc
On Thu, 2 Aug 2018 at 21:32, Moritz Cygorek wrote: > Hi, > > > I want to diagonalize a huge sparse matrix and I'm using the Kryov-Schur > method with harmonic extraction (command line option -eps_harmonic ) > implemented in SLEPc. > > > I manually distribute a sparse matrix across several CPUs and everything > works fine when: > > - I do _not_ use harmonic extraction > > - I use harmonic extraction on only a single CPU > > > If I try do use harmonic extraction on multiple CPUs, I get a memory > corruption. > > I'm not quite sure where to look at, but somewhere in the output, I find: > > > > [1]PETSC ERROR: PetscMallocValidate: error detected at > PetscSignalHandlerDefault() line 145 in > /home/applications/sources/libraries/petsc-3.9.3/src/sys/error/signal.c > [1]PETSC ERROR: Memory [id=0(9072)] at address 0x145bcd0 is corrupted > (probably write past end of array) > [1]PETSC ERROR: Memory originally allocated in DSAllocateWork_Private() > line 74 in > /home/applications/sources/libraries/slepc-3.9.2/src/sys/classes/ds/interface/dspriv.c > > > Now, I have the feeling that this might be a bug in SLEPc because, if I > had messed up the matrix initialization and distribution, I should also get > a memory corruption when I don't use harmonic extraction, right? > Not necessarily > Any suggestions what might be going on? > Run your code through valgrind and make sure that your application code is clean. See here http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind If errors are detected in your application code, fix them and see if the slepc errors go away. If your code is valgrind clean, send through the relevant chunk of the valgrind report indicating exactly where the error is occurring Thanks, Dave > Regards, > > Moritz > > > > > > > >
[petsc-users] memory corruption when using harmonic extraction with SLEPc
Hi, I want to diagonalize a huge sparse matrix and I'm using the Kryov-Schur method with harmonic extraction (command line option -eps_harmonic ) implemented in SLEPc. I manually distribute a sparse matrix across several CPUs and everything works fine when: - I do _not_ use harmonic extraction - I use harmonic extraction on only a single CPU If I try do use harmonic extraction on multiple CPUs, I get a memory corruption. I'm not quite sure where to look at, but somewhere in the output, I find: [1]PETSC ERROR: PetscMallocValidate: error detected at PetscSignalHandlerDefault() line 145 in /home/applications/sources/libraries/petsc-3.9.3/src/sys/error/signal.c [1]PETSC ERROR: Memory [id=0(9072)] at address 0x145bcd0 is corrupted (probably write past end of array) [1]PETSC ERROR: Memory originally allocated in DSAllocateWork_Private() line 74 in /home/applications/sources/libraries/slepc-3.9.2/src/sys/classes/ds/interface/dspriv.c Now, I have the feeling that this might be a bug in SLEPc because, if I had messed up the matrix initialization and distribution, I should also get a memory corruption when I don't use harmonic extraction, right? Any suggestions what might be going on? Regards, Moritz
Re: [petsc-users] Solving Linear Systems with Scalar Real and Complex
It looks like ML and hypre are working well now. If you want to debug GAMG you can run with -info, which is very noisy, and grep on GAMG or send the whole output. BTW, what equations are you solving? On Thu, Aug 2, 2018 at 5:12 AM Pierpaolo Minelli wrote: > Thank you very much for the correction. > By rebalancing the matrix coefficients, making them dimensionless, in my > complex problem i managed to obtain a better result with both ML and HYPRE. > GAMG instead seems to be unable to converge and in fact I get unexpected > results. I report the outputs of the three simulations again (i tried to > use also -pc_gamg_square_graph 20 without any improvement). > > Pierpaolo > > *-pc_type hypre -ksp_rtol 1.e-7 -ksp_monitor_true_residual* > > 0 KSP preconditioned resid norm 1.984853668904e-02 true resid norm > 2.979865703850e-03 ||r(i)||/||b|| 1.e+00 > 1 KSP preconditioned resid norm 1.924446712661e-04 true resid norm > 1.204260204811e-04 ||r(i)||/||b|| 4.041323752460e-02 > 2 KSP preconditioned resid norm 5.161509100765e-06 true resid norm > 2.810809726926e-06 ||r(i)||/||b|| 9.432672496933e-04 > 3 KSP preconditioned resid norm 9.297326931238e-08 true resid norm > 4.474617977876e-08 ||r(i)||/||b|| 1.501617328625e-05 > 4 KSP preconditioned resid norm 1.910271882670e-09 true resid norm > 9.637470658283e-10 ||r(i)||/||b|| 3.234196308186e-07 > 0 KSP preconditioned resid norm 2.157687745805e+04 true resid norm > 3.182001523188e+03 ||r(i)||/||b|| 1.e+00 > 1 KSP preconditioned resid norm 1.949268476386e+02 true resid norm > 1.243419788627e+02 ||r(i)||/||b|| 3.907665598415e-02 > 2 KSP preconditioned resid norm 5.078054475792e+00 true resid norm > 2.745355604400e+00 ||r(i)||/||b|| 8.627763325675e-04 > 3 KSP preconditioned resid norm 8.663802743529e-02 true resid norm > 4.254290979292e-02 ||r(i)||/||b|| 1.336985839979e-05 > 4 KSP preconditioned resid norm 1.795605563039e-03 true resid norm > 9.040507428245e-04 ||r(i)||/||b|| 2.841138623714e-07 > 0 KSP preconditioned resid norm 6.728304961395e+02 true resid norm > 1.879478105170e+02 ||r(i)||/||b|| 1.e+00 > 1 KSP preconditioned resid norm 2.190497539532e+01 true resid norm > 4.630095820203e+02 ||r(i)||/||b|| 2.463500802413e+00 > 2 KSP preconditioned resid norm 8.425561564252e-01 true resid norm > 7.012565302251e+01 ||r(i)||/||b|| 3.731123700223e-01 > 3 KSP preconditioned resid norm 3.029848345705e-02 true resid norm > 4.379018464663e+00 ||r(i)||/||b|| 2.329911932795e-02 > 4 KSP preconditioned resid norm 7.374025528575e-04 true resid norm > 1.337183702137e-01 ||r(i)||/||b|| 7.114654320570e-04 > 5 KSP preconditioned resid norm 3.009400175162e-05 true resid norm > 7.731135032616e-03 ||r(i)||/||b|| 4.113447776459e-05 > > *-pc_type ml -ksp_rtol 1.e-7 -ksp_monitor_true_residual* > > 0 KSP preconditioned resid norm 1.825767020538e-02 true resid norm > 2.979865703850e-03 ||r(i)||/||b|| 1.e+00 > 1 KSP preconditioned resid norm 6.495628259383e-04 true resid norm > 3.739440526742e-04 ||r(i)||/||b|| 1.254902367550e-01 > 2 KSP preconditioned resid norm 4.971875712015e-05 true resid norm > 2.118856024328e-05 ||r(i)||/||b|| 7.110575559127e-03 > 3 KSP preconditioned resid norm 3.726806462912e-06 true resid norm > 1.370355844514e-06 ||r(i)||/||b|| 4.598716790303e-04 > 4 KSP preconditioned resid norm 2.496898447120e-07 true resid norm > 9.494701893753e-08 ||r(i)||/||b|| 3.186285167645e-05 > 5 KSP preconditioned resid norm 1.541814409235e-08 true resid norm > 6.216753346769e-09 ||r(i)||/||b|| 2.086252859898e-06 > 6 KSP preconditioned resid norm 1.153890845138e-09 true resid norm > 5.290239857745e-10 ||r(i)||/||b|| 1.775328281039e-07 > 0 KSP preconditioned resid norm 1.961448048639e+04 true resid norm > 3.182001523188e+03 ||r(i)||/||b|| 1.e+00 > 1 KSP preconditioned resid norm 7.940694187370e+02 true resid norm > 4.331011797078e+02 ||r(i)||/||b|| 1.361096707691e-01 > 2 KSP preconditioned resid norm 6.141876183555e+01 true resid norm > 2.734346095946e+01 ||r(i)||/||b|| 8.593164007057e-03 > 3 KSP preconditioned resid norm 4.872887149917e+00 true resid norm > 1.774255242955e+00 ||r(i)||/||b|| 5.575909470898e-04 > 4 KSP preconditioned resid norm 3.088357668927e-01 true resid norm > 1.183728642343e-01 ||r(i)||/||b|| 3.720075662179e-05 > 5 KSP preconditioned resid norm 2.030016543073e-02 true resid norm > 7.924623904393e-03 ||r(i)||/||b|| 2.490452580442e-06 > 6 KSP preconditioned resid norm 1.458276941086e-03 true resid norm > 6.781128809522e-04 ||r(i)||/||b|| 2.131089114856e-07 > 0 KSP preconditioned resid norm 6.230592591466e+02 true resid norm > 1.879478105170e+02 ||r(i)||/||b|| 1.e+00 > 1 KSP preconditioned resid norm 3.825349376322e+01 true resid norm > 7.755266613308e+02 ||r(i)||/||b|| 4.126287287931e+00 > 2 KSP preconditioned resid norm 4.321561767174e+00 true resid norm > 2.170558998883e+02 ||r(i)||/||b|| 1.154873255992e+00 > 3 KSP preconditioned resid
Re: [petsc-users] Parallel QR decomposition
Hi,
sorry i haven't answered in a while.
>> i was wondering if there is an implementation of parallel qr
decomposition in PETSc.
> For what purpose do you need a QR? We may have suggestions if we know
how you want to use it.
I found an interesting algorithm for radial basis function interpolation.
It mitigates the ill-conditioning that arises from making radial basis
functions increasingly flat. The paper can be found at
https://amath.colorado.edu/faculty/fornberg/Docs/SISCmanuscript.pdf
The algorithm defines a Matrix C that is rectangular (more columns than
rows). Then a QR Decomposition of C is computed. It looks like this:
C = Q [R1 R2]
with R1 being rectangular and R2 being stacked on it horizontally. What i
want to compute in the end is:
R_1^{-1} R_2
Which isn't that hard to compute once R is known, as this is just backward
substitution. Anyway i still need to do a full QR decomposition (Even
though i don't actually need Q, it would be easy to calculate from R, so i
don't think there is any shortcut to Q, except that i don't have to store
Q).
What i was thinking of as a possible solution is to parallelize Givens
rotations by leveraging the fact that it only has local row-wise data
dependencies in each step.
Best regards
David Sommer
Re: [petsc-users] Solving Linear Systems with Scalar Real and Complex
Thank you very much for the correction. By rebalancing the matrix coefficients, making them dimensionless, in my complex problem i managed to obtain a better result with both ML and HYPRE. GAMG instead seems to be unable to converge and in fact I get unexpected results. I report the outputs of the three simulations again (i tried to use also -pc_gamg_square_graph 20 without any improvement). Pierpaolo -pc_type hypre -ksp_rtol 1.e-7 -ksp_monitor_true_residual 0 KSP preconditioned resid norm 1.984853668904e-02 true resid norm 2.979865703850e-03 ||r(i)||/||b|| 1.e+00 1 KSP preconditioned resid norm 1.924446712661e-04 true resid norm 1.204260204811e-04 ||r(i)||/||b|| 4.041323752460e-02 2 KSP preconditioned resid norm 5.161509100765e-06 true resid norm 2.810809726926e-06 ||r(i)||/||b|| 9.432672496933e-04 3 KSP preconditioned resid norm 9.297326931238e-08 true resid norm 4.474617977876e-08 ||r(i)||/||b|| 1.501617328625e-05 4 KSP preconditioned resid norm 1.910271882670e-09 true resid norm 9.637470658283e-10 ||r(i)||/||b|| 3.234196308186e-07 0 KSP preconditioned resid norm 2.157687745805e+04 true resid norm 3.182001523188e+03 ||r(i)||/||b|| 1.e+00 1 KSP preconditioned resid norm 1.949268476386e+02 true resid norm 1.243419788627e+02 ||r(i)||/||b|| 3.907665598415e-02 2 KSP preconditioned resid norm 5.078054475792e+00 true resid norm 2.745355604400e+00 ||r(i)||/||b|| 8.627763325675e-04 3 KSP preconditioned resid norm 8.663802743529e-02 true resid norm 4.254290979292e-02 ||r(i)||/||b|| 1.336985839979e-05 4 KSP preconditioned resid norm 1.795605563039e-03 true resid norm 9.040507428245e-04 ||r(i)||/||b|| 2.841138623714e-07 0 KSP preconditioned resid norm 6.728304961395e+02 true resid norm 1.879478105170e+02 ||r(i)||/||b|| 1.e+00 1 KSP preconditioned resid norm 2.190497539532e+01 true resid norm 4.630095820203e+02 ||r(i)||/||b|| 2.463500802413e+00 2 KSP preconditioned resid norm 8.425561564252e-01 true resid norm 7.012565302251e+01 ||r(i)||/||b|| 3.731123700223e-01 3 KSP preconditioned resid norm 3.029848345705e-02 true resid norm 4.379018464663e+00 ||r(i)||/||b|| 2.329911932795e-02 4 KSP preconditioned resid norm 7.374025528575e-04 true resid norm 1.337183702137e-01 ||r(i)||/||b|| 7.114654320570e-04 5 KSP preconditioned resid norm 3.009400175162e-05 true resid norm 7.731135032616e-03 ||r(i)||/||b|| 4.113447776459e-05 -pc_type ml -ksp_rtol 1.e-7 -ksp_monitor_true_residual 0 KSP preconditioned resid norm 1.825767020538e-02 true resid norm 2.979865703850e-03 ||r(i)||/||b|| 1.e+00 1 KSP preconditioned resid norm 6.495628259383e-04 true resid norm 3.739440526742e-04 ||r(i)||/||b|| 1.254902367550e-01 2 KSP preconditioned resid norm 4.971875712015e-05 true resid norm 2.118856024328e-05 ||r(i)||/||b|| 7.110575559127e-03 3 KSP preconditioned resid norm 3.726806462912e-06 true resid norm 1.370355844514e-06 ||r(i)||/||b|| 4.598716790303e-04 4 KSP preconditioned resid norm 2.496898447120e-07 true resid norm 9.494701893753e-08 ||r(i)||/||b|| 3.186285167645e-05 5 KSP preconditioned resid norm 1.541814409235e-08 true resid norm 6.216753346769e-09 ||r(i)||/||b|| 2.086252859898e-06 6 KSP preconditioned resid norm 1.153890845138e-09 true resid norm 5.290239857745e-10 ||r(i)||/||b|| 1.775328281039e-07 0 KSP preconditioned resid norm 1.961448048639e+04 true resid norm 3.182001523188e+03 ||r(i)||/||b|| 1.e+00 1 KSP preconditioned resid norm 7.940694187370e+02 true resid norm 4.331011797078e+02 ||r(i)||/||b|| 1.361096707691e-01 2 KSP preconditioned resid norm 6.141876183555e+01 true resid norm 2.734346095946e+01 ||r(i)||/||b|| 8.593164007057e-03 3 KSP preconditioned resid norm 4.872887149917e+00 true resid norm 1.774255242955e+00 ||r(i)||/||b|| 5.575909470898e-04 4 KSP preconditioned resid norm 3.088357668927e-01 true resid norm 1.183728642343e-01 ||r(i)||/||b|| 3.720075662179e-05 5 KSP preconditioned resid norm 2.030016543073e-02 true resid norm 7.924623904393e-03 ||r(i)||/||b|| 2.490452580442e-06 6 KSP preconditioned resid norm 1.458276941086e-03 true resid norm 6.781128809522e-04 ||r(i)||/||b|| 2.131089114856e-07 0 KSP preconditioned resid norm 6.230592591466e+02 true resid norm 1.879478105170e+02 ||r(i)||/||b|| 1.e+00 1 KSP preconditioned resid norm 3.825349376322e+01 true resid norm 7.755266613308e+02 ||r(i)||/||b|| 4.126287287931e+00 2 KSP preconditioned resid norm 4.321561767174e+00 true resid norm 2.170558998883e+02 ||r(i)||/||b|| 1.154873255992e+00 3 KSP preconditioned resid norm 2.806349660119e-01 true resid norm 2.931496742303e+01 ||r(i)||/||b|| 1.559739767247e-01 4 KSP preconditioned resid norm 1.707567397460e-02 true resid norm 2.684913551118e+00 ||r(i)||/||b|| 1.428542074383e-02 5 KSP preconditioned resid norm 1.741069717192e-03 true resid norm 2.914639372698e-01 ||r(i)||/||b|| 1.550770591410e-03 6 KSP preconditioned resid norm
