In my case N=M but n for process 0, 1, 2, 3,... no_processes-1 can be different
from the nth process like in the example where the nth process=Proc2 and has
only two rows while all other processes have three rows:
Example from the PETSc webpage mentioned before:
1 2 0 | 0 3 0 | 0 4
Proc0 0 5 6 | 7 0 0 | 8 0
9 0 10 | 11 0 0 | 12 0
-------------------------------------
13 0 14 | 15 16 17 | 0 0
Proc1 0 18 0 | 19 20 21 | 0 0
0 0 0 | 22 23 0 | 24 0
-------------------------------------
Proc2 25 26 27 | 0 0 28 | 29 0
30 0 0 | 31 32 33 | 0 34and I need to enter different values
for d_nnz and o_nnz for each row somewhere too
proc0: d_nnz = [2,2,2] and o_nnz = [2,2,2]
proc1: d_nnz = [3,3,2] and o_nnz = [2,1,1]
proc2: d_nnz = [1,1] and o_nnz = [4,4]
I simply can't identify the function(s) used to set the values for n, d_nnz and
o_nnz for the individual local matrices allocated to all the processes if n
isn't the same for all processes and d_nnz and o_nnz are different for each
local matrix?
Approach described on the PETSc webpage:
MatCreate(...,&A);
MatSetType(A,MATMPIAIJ);
MatSetSizes(A, m,n,M,N); // for the example above using this function would
set the no. of rows for Proc2 to 3 but it's 2
MatMPIAIJSetPreallocation(A,...); // this function can be used to set
values for ONE local matrix only
In addition to that I don't know which functions to use to preallocate memory
for ALL local matrices when each of them has different values for d_nnz and
o_nnz.
I other words, what's the code for the 3 process example above? (entering the
matrix structure and allocating memory)
Klaus
Am Freitag, 20. April 2018, 17:13:26 MESZ hat Smith, Barry F.
<[email protected]> Folgendes geschrieben:
For square matrices almost always n is the same as m. On different processes
m can be different. You get to decide what makes sense for each processes what
its m should be.
Barry
> On Apr 20, 2018, at 10:05 AM, Klaus Burkart <[email protected]> wrote:
>
> I think I understood the matrix structure for parallel computation with the
> rows, diagonal (d) and off-diagonal (o) structure, where I have problems is
> how to do the setup including memory allocation in PETSc:
>
> Lets assume, I use a 16 core workstation (=16 processes) and the number of
> nonzeros varies in each row for both d and o and the number of rows assigned
> to each process differs too - at least for the nth process.
>
> Looking at the manual and
> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateAIJ.html#MatCreateAIJ,
> I don't understand how to enter a global matrix when n is NOT the same for
> each process as e.g. in MatSetSizes(A, m,n,M,N); n and m are integers, not
> arrays?
>
> MatCreate(...,&A);
>
> MatSetType(A,MATMPIAIJ);
>
> MatSetSizes(A, m,n,M,N); // seems to assume n and m are the same for each
>process which isn't even the case in the example on the page mentioned above?!
>
> MatMPIAIJSetPreallocation(A,...);
>
>
> How can I enter the parallel global-local matrix structure?
>
> How can the memory preallocation be done?
>
> Klaus
>
> Am Donnerstag, 19. April 2018, 01:47:59 MESZ hat Smith, Barry F.
> <[email protected]> Folgendes geschrieben:
>
>
>
>
> > On Apr 18, 2018, at 4:42 PM, [email protected] wrote:
> >
> > So, practically speaking, l should invent routines to decompose the matrix
> > e.g. into a block matrix structure to be able to make real use of PETSc ie.
> > be able to solve a linear system using more than one process/core?
>
> To really use PETSc efficiently/effectively you need to generate your matrix
>in parallel.
>
> Barry
>
> >
> > Klaus
> >
> > Von meinem Huawei-Mobiltelefon gesendet
> >
> >
> > -------- Originalnachricht --------
> > Betreff: Re: [petsc-users] Matrix and vector type selection & memory
> > allocation for efficient matrix import?
> > Von: "Smith, Barry F."
> > An: Klaus Burkart
> > Cc: PETSc Users List
> >
> >
> >
> > If you can only generate the nonzero allocation sequentially you can only
> > solve sequentially which means your matrix is MATSEQAIJ and your vector is
> > VECSEQ and your communicator is PETSC_COMM_SELF.
> >
> > If you pass and array for nnz, what you pass for nz is irrelevant, you
> > might as well pass 0.
> >
> > Barry
> >
> >
> > > On Apr 18, 2018, at 10:48 AM, Klaus Burkart wrote:
> > >
> > > More questions about matrix and vector type selection for my application:
> > >
> > > My starting point is a huge sparse matrix which can be symmetric or
> > > asymmetric and a rhs vector. There's no defined local or block structure
> > > at all, just row and column indices and the values and an array style rhs
> > > vector together describing the entire linear system to be solved. With
> > > quite some effort, I should be able to create an array nnz[N] containing
> > > the number of nonzeros per row in the global matrix for memory allocation
> > > which would leave me with MatSeqAIJSetPreallocation(M, 0, nnz); as the
> > > only option for efficient memory allocation ie. a MATSEQAIJ matrix and
> > > VECSEQ. I assume here, that 0 indicates different numbers of nonzero
> > > values in each row, the exact number being stored in the nnz array.
> > > Regarding this detail but one example assume a constant number of nz per
> > > row so I am not sure whether I should write 0 or NULL for nz?
> > >
> > > I started with:
> > >
> > > MatCreate(PETSC_COMM_WORLD, &M);
> > > MatSetSizes(M, PETSC_DECIDE, PETSC_DECIDE, N, N);
> > > MatSetFromOptions(M);
> > >
> > > taken from a paper and assume, the latter would set the matrix type to
> > > MATSEQAIJ which might conflict with PETSC_COMM_WORLD. Maybe
> > > decompositioning took place at an earlier stage and the authors of the
> > > paper were able to retrieve the local data and structure.
> > >
> > > What type of matrix and vector should I use for my application e.g.
> > > MATSEQAIJ and VECSEQ to be able to use MatSeqAIJSetPreallocation(M, 0,
> > > nnz); for efficient memory allocation?
> > >
> > > In this case, where would the decompositioning / MPI process allocation
> > > take place?
> >
