Re: [petsc-users] An issue of extraction of factorization matrices in sparse direct solver

There are some utility routines for printing L\U in superlu_dist: SRC/dutil_dist.c 1. You can output the L factor to a file with the triplet format by using https://github.com/xiaoyeli/superlu_dist/blob/324d65fced6ce8abf0eb900223cba0207d538db7/SRC/dutil_dist.c#L675 but use line 755

Re: [petsc-users] [EXTERNAL] Re: Question about SuperLU

e the condition number through petsc and it is > reasonable. > > Qi > > > On Jun 10, 2022, at 6:35 PM, Xiaoye S. Li wrote: > > -mat_superlu_dist_replacetinypivot > >

Re: [petsc-users] Question about SuperLU

On Fri, Jun 10, 2022 at 7:43 AM Tang, Qi wrote: > We use superlu_dist. > > We have a 2 x 2 block where directly calling suplerlu_dist fails, but a pc > based on a fieldsplit Schur complement + superlu_dist on the assembled > Schur complement matrix converges. (All the converge criteria are

Re: [petsc-users] [EXTERNAL] Re: Question about SuperLU

Could that be due to "numerical zero pivot" (caused due to cancellation and underflow)? You can try to force diagonal to be nonzero. Looking at the options page: https://petsc.org/main/docs/manualpages/Mat/MATSOLVERSUPERLU_DIST/ You can enable this one: -mat_superlu_dist_replacetinypivot

Re: [petsc-users] Question about SuperLU

Are you using serial SuperLU, or distributed-memory SuperLU_DIST? What are the algorithm options are you using? Sherry Li On Thu, Jun 9, 2022 at 2:20 PM Jorti, Zakariae via petsc-users < petsc-users@mcs.anl.gov> wrote: > Hi, > > I am solving non-linear problem that has 5 unknowns {ni, T, E, B,

Re: [petsc-users] Downloaded superlu_dist could not be used. Please check install in $PREFIX

There was a merge error in the master branch. I fixed it today. Not sure whether that's causing your problem. Can you try now? Sherry On Mon, Jan 17, 2022 at 11:55 AM Fande Kong wrote: > I am trying to port PETSc-3.16.3 to the MOOSE ecosystem. I got an error > that PETSc could not build

Re: [petsc-users] Combining SUPERLU_DIST with CUDA fails

Seems -isystem is a gcc option. Can you try not to give "-I/usr/local/cuda-11.5/include" in CUDA_ARCH_FLAGS, see whether nvcc can find the proper include/ directory? Sherry On Tue, Dec 21, 2021 at 10:26 AM Roland Richter wrote: > I'm already using GCC, together with Intel MPI, thus that should

Re: [petsc-users] MatDestroy problem with multiple matrices and SUPERLU_DIST

When you have two matrices, I think you should create two separate communicators, i.e., two superlu grids, Sherry Li On Wed, Apr 21, 2021 at 6:08 AM Deij-van Rijswijk, Menno wrote: > Good afternoon, > > > > In our code we're using two matrices that are preconditioned using > SUPERLU_dist. Upon

Re: [petsc-users] superlu_dist segfault

5,520 >> alloc'd >> ==43569==at 0x4C2D814: memalign (vg_replace_malloc.c:906) >> ==43569==by 0x4C2D97B: posix_memalign (vg_replace_malloc.c:1070) >> ==43569==by 0x1464D488: superlu_malloc_dist (memory.c:127) >> ==43569==by 0x1473C451: pzgstrs (pzgstrs

Re: [petsc-users] superlu_dist segfault

Hong: thanks for the diagnosis! Marius: how many OpenMP threads are you using per MPI task? In an earlier email, you mentioned the allocation failure at the following line: if ( !(lsum = (doublecomplex*) SUPERLU_MALLOC(sizelsum*num_thread * sizeof(doublecomplex ABORT("Malloc fails for

Re: [petsc-users] SUPERLU_DIST in single precision

> > Sherry prepared. > > > > > >> On Jul 27, 2020, at 2:10 PM, f...@rzg.mpg.de wrote: > >> > >> Hi Shery, > >> Yes, ideally we would like to compile PETSc in single precision and > >> simply run a single precision version of SUPERLU

Re: [petsc-users] SUPERLU_DIST in single precision

Barry, I have a branch 'Mixed-precision' working with single precision FP32. I assume Felix wants to use superlu_dist from petsc. How do you want to incorporate it in petsc? https://github.com/xiaoyeli/superlu_dist PS1: in this version, FP32 only works on CPU. FP64 and complex-FP64 all work

Re: [petsc-users] SUPERLU_DIST in single precision

I can package a CPU version over the weekend. Sherry On Wed, Jul 22, 2020 at 6:04 AM wrote: > Hi Barry, > for now I just want to run everything in single on CPUs only with > SUPERLU_DIST. Maybe we will also incorporate GPUs in the future, but > there are no immediate plans yet. So if you could

Re: [petsc-users] SUPERLU_DIST in single precision

Yes. We will wait for his reply. -Sherry On Tue, Jul 21, 2020 at 6:30 PM Barry Smith wrote: > > Thanks Sherry. But the CPU version runs correctly right so if Felix is > only interested in CPUs then I could give it to him now? > > Barry > > > On Jul 21, 2020,

Re: [petsc-users] SUPERLU_DIST in single precision

Barry, We have a macro-ized basefile to produced all 4 precisions for SuperLU_DIST. In the past, we didn't generate single precision code, since there was no request. Since now we started working on mixed precision algorithms. I generated single precision code a few weeks ago, and fixed some

Re: [petsc-users] SuperLU_Dist in single

We plan to release single precision version soon. -Sherry On Wed, Jun 17, 2020 at 9:11 AM Satish Balay wrote: > Its just double precision. > > cc: Sherry > > Satish > > On Wed, 17 Jun 2020, f...@rzg.mpg.de wrote: > > > Dear PETSc Team, > > is it just me or is SuperLU_Dist only available in

Re: [petsc-users] SuperLU + GPUs

d. > > If anyone is interested in looking at this, I was also able to see that > plex/ex10 in my branch, which is a very simple test , also does not crash > and also does not seem to use GPUs in SuperLU. > > > On Sat, Apr 18, 2020 at 11:46 AM Xiaoye S. Li wrote: > >> Whe

Re: [petsc-users] SuperLU + GPUs

When you install "-download-superlu_dist", that is from 'master' branch? In the error trace, I recognized this: > [h50n09:102287] [ 9] /ccs/home/adams/petsc/arch- summit-opt-gnu-cuda-omp/lib/libsuperlu_dist.so.6(dDestroy_ LU+0xc4)[0x2195aff4] This is to free the L and U data structures at

Re: [petsc-users] superlu_dist+parmetis

I am pretty sure this is a bug in parmetis. A few years ago, I nailed down this "divide by zero" bug, and reported to them. They said they would take a look, but never did. This usually happens when the graph is relatively dense. You can try sequential Metis. Sherry On Thu, Feb 14, 2019 at

Re: [petsc-users] SuperLU_dist bug with parallel symbolic factorisation

Is it possible to download this particular matrix, so I can do standalone investigation? Sherry On Tue, May 22, 2018 at 12:22 PM, Eric Chamberland < eric.chamberl...@giref.ulaval.ca> wrote: > Hi Fande, > > I don't know, I am working and validating with a DEBUG version of PETSc, > and this "mwe"

Re: [petsc-users] SuperLU_dist bug with parallel symbolic factorisation

Numerical factorization is always parallel (based on number of MPI tasks and OMP_NUM_THREADS you set), the issue here is only related to symbolic factorization (figuring out the nonzero pattern in the LU factors). Default setting is to use sequential symbolic factorization, precisely due to the

Re: [petsc-users] SuperLU_dist bug with parallel symbolic factorisation

Indeed, I am pretty sure the bug is in ParMETIS. A few years ago, I sent a sample matrix and debug trace to George Karypis, he was going to look at it, but never did. This bug seems to show up when the graph is relatively dense. Can you try to use serial symbolic factorization and Metis?

Re: [petsc-users] PDSLin 2.0.0 example memory corruption error

This looks like error from ParMETIS -- some processes do not have initial vertices. Which version of PerMETIS are you using? There may be error in the way you run the test: $ mpiexec -n 8 ./dtest input matrices/U.dat Try to remove "input" in the command line. Can also try using smaller number

Re: [petsc-users] Solving NON-Diagonally dominant sparse system

If you need to use SuperLU_DIST, the pivoting is done statically, using maximum weighted matching, so the small diagonals are usually taken care as well. It is not as good as partial pivoting, but works most of the time. Sherry On Mon, Apr 10, 2017 at 12:07 PM, Barry Smith

Re: [petsc-users] Error with SuperLU_DIST (mkl related?)

I just updated version string in git repo and tarball. Sherry On Sat, Dec 31, 2016 at 10:39 AM, Satish Balay wrote: > Ok - one more place superlu_dist stores version number - that needs > updating with every release. > > cc:ing Sherry > > Satish > > On Sat, 31 Dec 2016, Eric

Re: [petsc-users] SuperLU_dist issue in 3.7.4

Some graph preprocessing steps can be skipped ONLY IF a previous factorization was done, and the information can be reused (AS INPUT) to the new factorization. In general, the driver routine SRC/pdgssvx.c() performs the LU factorization of the following (preprocessed) matrix:

Re: [petsc-users] SuperLU_dist issue in 3.7.4

> [0]PETSC ERROR: [0] MatLUFactorNumeric_SuperLU_DIST line 282 >>>> /home/anton/LIB/petsc/src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c >>>> [0]PETSC ERROR: [0] MatLUFactorNumeric line 2985 >>>> /home/anton/LIB/petsc/src/mat/interface/matrix.c >>>

Re: [petsc-users] SuperLU_dist issue in 3.7.4

Which version of superlu_dist does this capture? I looked at the original error log, it pointed to pdgssvx: line 161. But that line is in comment block, not the program. Sherry On Mon, Oct 10, 2016 at 7:27 AM, Anton Popov wrote: > > > On 10/07/2016 05:23 PM, Satish

Re: [petsc-users] SuperLU convergence problem (More test)

Hi Hong, Thank you very much for keeping tracking down the problem. I apologize for slow response... some administrative duties consumed most of my cycles for a few days. SamePattern can be used for a sequence of linear systems with the same sparsity pattern. Then the sparsity ordering is done

Re: [petsc-users] SuperLU MPI-problem

not using I-node routines Norm of error 5.21214e-15 iterations 1 Mahir From: Hong [mailto:hzh...@mcs.anl.gov] Sent: den 6 augusti 2015 16:36 To: Ülker-Kaustell, Mahir Cc: Hong; Xiaoye S. Li; PETSc users list Subject: Re: [petsc-users] SuperLU MPI-problem Mahir: I have

Re: [petsc-users] SuperLU MPI-problem

, then configure petsc with '--download-superlu_dist=superlu_dist_4.1.tar.gz' 3. build petsc Let us know if the issue remains. Hong -- Forwarded message -- From: *Xiaoye S. Li* x...@lbl.gov Date: Wed, Jul 29, 2015 at 2:24 PM Subject: Fwd: [petsc-users] SuperLU MPI-problem

Re: [petsc-users] Can't expand MemType 1: jcol 16104

/make.xc30 SRC/int_t SRC/lnbrow SRC/make.out SRC/rnbrow SRC/temp SRC/temp1 Thanks, Satish On Tue, 28 Jul 2015, Xiaoye S. Li wrote: I am checking v4.1 now. I'll let you know when I fixed the problem. Sherry On Tue, Jul 28, 2015 at 8:27 AM, Hong hzh

Re: [petsc-users] Can't expand MemType 1: jcol 16104

] != MPI_REQUEST_NULL) { -- MPI_Wait (recv_req[0], status); We will update petsc interface to superlu_dist v4.1. Hong On Mon, Jul 27, 2015 at 11:33 PM, Xiaoye S. Li x...@lbl.gov wrote: ​Hong, Thanks for trying out. The extra printings are not properly guarded by the print

Re: [petsc-users] Can't expand MemType 1: jcol 16104

, Xiaoye S. Li wrote: Indeed, the parallel symbolic factorization routine needs power of 2 processes, however, you can use however many processes you need; internally, we redistribute matrix to nearest power of 2 processes, do symbolic, then redistribute back to all the processes to do

Re: [petsc-users] SuperLU MPI-problem

010 452 30 82, mahir.ulker-kaust...@tyrens.se __ -Original Message- From: Barry Smith [mailto:bsm...@mcs.anl.gov] Sent: den 22 juli 2015 02:57 To: Ülker-Kaustell, Mahir Cc: Xiaoye S. Li; petsc-users Subject: Re: [petsc-users] SuperLU MPI

Re: [petsc-users] SuperLU MPI-problem

The default SuperLU_DIST setting is to serial symbolic factorization. Therefore, what matters is how much memory do you have per MPI task? The code failed to malloc memory during redistribution of matrix A to {L\U} data struction (using result of serial symbolic factorization.) You can use

Re: [petsc-users] Can't expand MemType 1: jcol 16104

Did you find out how to change option to use parallel symbolic factorization? Perhaps PETSc team can help. Sherry On Tue, Jul 7, 2015 at 3:58 PM, Xiaoye S. Li x...@lbl.gov wrote: Is there an inquiry function that tells you all the available options? Sherry On Tue, Jul 7, 2015 at 3:25 PM

Re: [petsc-users] Can't expand MemType 1: jcol 16104

be some blocks where the pivot would be zero, hence creating the FPE?? Thanks, Anthony On Wed, Jul 8, 2015 at 6:46 AM, Xiaoye S. Li x...@lbl.gov wrote: Did you find out how to change option to use parallel symbolic factorization? Perhaps PETSc team can help. Sherry On Tue, Jul 7

Re: [petsc-users] Can't expand MemType 1: jcol 16104

? is it -mat_superlu_dist_matinput 1 Thanks, Anthony On Tue, Jul 7, 2015 at 3:08 PM, Xiaoye S. Li x...@lbl.gov wrote: For superlu_dist failure, this occurs during symbolic factorization. Since you are using serial symbolic factorization, it requires the entire graph of A to be available in the memory

Re: [petsc-users] Can't expand MemType 1: jcol 16104

For superlu_dist failure, this occurs during symbolic factorization. Since you are using serial symbolic factorization, it requires the entire graph of A to be available in the memory of one MPI task. How much memory do you have for each MPI task? It won't help even if you use more processes.

Re: [petsc-users] Error using MUMPS to solve large linear system

in my case. Can you recommend any other parameters of petsc, superlu_dist, or mumps that I should try changing? I don't care in the end whether I use superlu_dist or mumps. Thanks! Matt Landreman On Tue, Feb 25, 2014 at 3:50 PM, Xiaoye S. Li x...@lbl.gov wrote: Very good! Thanks

Re: [petsc-users] Error using MUMPS to solve large linear system

,MATSOLVERSUPERLU_DIST); KSPSetFromOptions(ksp); PCSetFromOptions(pc); KSPSolve(ksp,b,x); All of these successfully return *except* the very last one to KSPSolve. Any help would be appreciated. Thanks! Samar On Feb 24, 2014, at 3:58 PM, Xiaoye S. Li x...@lbl.gov wrote: Samar: If you include

Re: [petsc-users] Error using MUMPS to solve large linear system

The hardware (http://www2.cisl.ucar.edu/resources/yellowstone/hardware) specs are: 2 GB/core, 32 GB/node (27 GB usable), (16 cores per node) I've been running on 8 nodes (so 8 x 27 ~ 216 GB). Thanks again for your help! Samar On Feb 25, 2014, at 1:00 PM, Xiaoye S. Li x...@lbl.gov wrote: I didn't

Re: [petsc-users] Error using MUMPS to solve large linear system

Samar: If you include the error message while crashing using superlu_dist, I probably know the reason. (better yet, include the printout before the crash. ) Sherry On Mon, Feb 24, 2014 at 9:56 AM, Hong Zhang hzh...@mcs.anl.gov wrote: Samar : There are limitations for direct solvers. Do not

Re: [petsc-users] ILUTP

The sequential SuperLU has ILUTP implementation, not in parallel versions. PETSc already supports the option of using SuperLU. In SuperLU distribution: EXAMPLE/zitersol.c : an example to use GMRES with ILUTP preconditioner (returned from driver SRC/zgsisx.c) SRC/zgsitrf.c : the actual ILUTP

[petsc-users] -mat_superlu_lwork

Yes, that limits your preallocated work array to be 2GB. Alternatively, you can set lwork=0, rely on superlu to do memory allocation for you. Then you can use more than 2GB memory. Sherry Li On Tue, Jul 3, 2012 at 9:32 AM, Satish Balay balay at mcs.anl.gov wrote: Actually its INT_MAX. From

[petsc-users] Superlu based ILUT report Zero pivot

The default settings in ILUTP of superlu is conservative, probably with less dropping than people want. The 2 most relevant parameters to control this are: ILU_DropTol : (default 1e-4), you can try to set it a larger, e.g., 0.01 ILU_FillFactor : (default 10), this allows about a

[petsc-users] Block ILU with AIJ matrix?

The ILUTP in SuperLU (4.2 and up) exploits blocking (supernodes) in the (approximate) L U factors. Note the block boundary is discovered on the fly, which is usually larger than the block size from the input matrix A. So the efficiency is pretty good. It uses threshold dropping with partial

[petsc-users] Petsc-3.2 with superlu_dist building error

The error is related to the new SuperLU_DIST (v3.0), not serial SuperLU. In PETSc / SuperLU_DIST wrapper: superlu_dist.c, line 634, DOUBLE should be replaced by SLU_DOUBLE. Sherry On Thu, Oct 27, 2011 at 11:44 AM, Jed Brown jedbrown at mcs.anl.gov wrote: On Thu, Oct 27, 2011 at 12:34, Ping

[petsc-users] Petsc-3.2 with superlu_dist building error

Okay, the older superlu_4.2.tar.gz is reverted, and the new one is superlu_4.3.tar.gz. Sherry On Thu, Oct 27, 2011 at 12:20 PM, Jed Brown jedbrown at mcs.anl.gov wrote: On Thu, Oct 27, 2011 at 13:11, Xiaoye S. Li xsli at lbl.gov wrote: The error is related to the new SuperLU_DIST (v3.0