On Thu, Mar 31, 2011 at 12:38 PM, gouarin <loic.gouarin at math.u-psud.fr>wrote:
> On 31/03/2011 19:06, Matthew Knepley wrote: > > On Thu, Mar 31, 2011 at 10:10 AM, gouarin <loic.gouarin at > math.u-psud.fr>wrote: > > Thanks for the reply. > > > On 31/03/2011 16:48, Hong Zhang wrote: > > Loic, > > I want to do the same for MATMPI. This is what I do: > > MatGetLocalMat(A,MAT_INITIAL_MATRIX,&Aloc); > MatGetRowIJ(Aloc, 1, PETSC_FALSE, PETSC_FALSE,&(Acsr.n),&(Acsr.ia), > &(Acsr.ja),&done); > MatGetArray(Aloc,&(Acsr.a)); > > Now, I need to know on what process the non local points are. > > Each process get its own local submatrix. See > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatCreateMPIAIJ.html > on data structure of MPIAIJ format. Applying your statements above to > the example listed on the page (a 8x8 matrix), Proc0 will get > Aloc=A(0:2,:), ..., Porc2 gets Aloc=A(6:7,:). > > Is there a petsc function that gives this partition ? > > MatGetOwnershipRange() > > I use this function to know the local points index. In the same way, with > this function I know the non local index. > > But this function doesn't give the process. I give you an example. I have > this matrix: > > 1 0 | 0 1 > 1 2 | 1 0 > ----------- > 0 0 | 1 0 > 0 2 | 0 1 > > I want this kind of format > > for proc 0 > a = {1,1,1,2,1} > ia = {0,2,5} > ja = {0,3,0,1,2} > rank = {0,0,1,1} > > for proc 1 > a = {1,2,1} > ia = {0,1,3} > ja = {0,2,1} > rank = {1,1,0} > > rank give me the process for each non zero column. Local rows should have > numbers 1,...N and offdiagonal entries with j>N. > > > The ownership range is the row partition. However, MPIAIJ matrices are > not stored as you indicate above. > > Different storage formats make the process you describe fragile. WHy do > you need the CSR form of the matrix? If you really > do, use MatGetSubMatrix() to get a SEQAIJ matrix on each process and pull > out the values. > > I want to use this format because I try to create a new multigrid PC > with this method: > > http://homepages.ulb.ac.be/~ynotay/AGMG/userguide/index.html > > I don't understand why it's not possible to have the process partition. > When you create a MPIAIJ matrix and you use a KSP to solve the system, you > know what values to exchange. No? Where is this information ? > 1) It is possible to get the partition of rows across processes. This is EXACTLY what MatGetOwnershipRange() returns, as I said in the last email. 2) As I explained, matrices can be stored in a variety of formats. The MPIAIJ format stores 2 local matrices per partition in order to overlap communication with computation. I can also use MatGetOwnershipRange() and MPI_Allgather to have this > information. > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetOwnershipRanges.html Matt > Loic > > > > > Matt > > > I suppose that I have no information about the structure of MPIAIJ. > > Loic > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > > -- > Loic Gouarin > Laboratoire de Math?matiques > Universit? Paris-Sud > B?timent 425 > 91405 Orsay Cedex > France > Tel: (+33) 1 69 15 60 14 > Fax: (+33) 1 69 15 67 18 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20110331/0528ef47/attachment.htm>
