On Fri, Apr 20, 2018 at 11:05 AM, Klaus Burkart <[email protected]> wrote:
> I think I understood the matrix structure for parallel computation with > the rows, diagonal (d) and off-diagonal (o) structure, where I have > problems is how to do the setup including memory allocation in PETSc: > > Lets assume, I use a 16 core workstation (=16 processes) and the number of > nonzeros varies in each row for both d and o and the number of rows > assigned to each process differs too - at least for the nth process. > > Looking at the manual and http://www.mcs.anl.gov/petsc/petsc-current/docs/ > manualpages/Mat/MatCreateAIJ.html#MatCreateAIJ, I don't understand how to > enter a global matrix when n is NOT the same for each process as e.g. in > MatSetSizes(A, m,n,M,N); n and m are integers, not arrays? > This is SPMD programming, so the call on each process gives a different n, and collectively you have 16 different n values. "Using MPI" is a great book for understanding this. Thanks, Matt > MatCreate(...,&A); > > MatSetType(A,MATMPIAIJ); > > MatSetSizes(A, m,n,M,N); // seems to assume n and m are the same for > each process which isn't even the case in the example on the page mentioned > above?! > > MatMPIAIJSetPreallocation(A,...); > > > How can I enter the parallel global-local matrix structure? > > How can the memory preallocation be done? > > Klaus > > Am Donnerstag, 19. April 2018, 01:47:59 MESZ hat Smith, Barry F. < > [email protected]> Folgendes geschrieben: > > > > > > On Apr 18, 2018, at 4:42 PM, [email protected] wrote: > > > > So, practically speaking, l should invent routines to decompose the > matrix e.g. into a block matrix structure to be able to make real use of > PETSc ie. be able to solve a linear system using more than one process/core? > > To really use PETSc efficiently/effectively you need to generate your > matrix in parallel. > > Barry > > > > > Klaus > > > > Von meinem Huawei-Mobiltelefon gesendet > > > > > > -------- Originalnachricht -------- > > Betreff: Re: [petsc-users] Matrix and vector type selection & memory > allocation for efficient matrix import? > > Von: "Smith, Barry F." > > An: Klaus Burkart > > Cc: PETSc Users List > > > > > > > > If you can only generate the nonzero allocation sequentially you can > only solve sequentially which means your matrix is MATSEQAIJ and your > vector is VECSEQ and your communicator is PETSC_COMM_SELF. > > > > If you pass and array for nnz, what you pass for nz is irrelevant, you > might as well pass 0. > > > > Barry > > > > > > > On Apr 18, 2018, at 10:48 AM, Klaus Burkart wrote: > > > > > > More questions about matrix and vector type selection for my > application: > > > > > > My starting point is a huge sparse matrix which can be symmetric or > asymmetric and a rhs vector. There's no defined local or block structure at > all, just row and column indices and the values and an array style rhs > vector together describing the entire linear system to be solved. With > quite some effort, I should be able to create an array nnz[N] containing > the number of nonzeros per row in the global matrix for memory allocation > which would leave me with MatSeqAIJSetPreallocation(M, 0, nnz); as the only > option for efficient memory allocation ie. a MATSEQAIJ matrix and VECSEQ. I > assume here, that 0 indicates different numbers of nonzero values in each > row, the exact number being stored in the nnz array. Regarding this detail > but one example assume a constant number of nz per row so I am not sure > whether I should write 0 or NULL for nz? > > > > > > I started with: > > > > > > MatCreate(PETSC_COMM_WORLD, &M); > > > MatSetSizes(M, PETSC_DECIDE, PETSC_DECIDE, N, N); > > > MatSetFromOptions(M); > > > > > > taken from a paper and assume, the latter would set the matrix type to > MATSEQAIJ which might conflict with PETSC_COMM_WORLD. Maybe > decompositioning took place at an earlier stage and the authors of the > paper were able to retrieve the local data and structure. > > > > > > What type of matrix and vector should I use for my application e.g. > MATSEQAIJ and VECSEQ to be able to use MatSeqAIJSetPreallocation(M, 0, > nnz); for efficient memory allocation? > > > > > > In this case, where would the decompositioning / MPI process > allocation take place? > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>
