Hi Hong, I think what you said about the ISs for MatGetSubMatrices() are right.
Today I tried to run my program to get submatrix by MatGetSubMatrices() twice. One time with ISs created with PETSC_COMM_WORLD and the other time with PETSC_COMM_SELF. I was creating a 6x3 submatrix on each processor from a 570x3 parallel dense matrix. There are only 2 processors in total. The results are as follows: ======== submatrix with ISs created with PETSC_COMM_WORLD ========= Processor 0 Mat Object: 1 MPI processes type: seqdense 2.7074996615625420e+10 6.8399452804377966e+11 -1.2730861976538324e+08 2.7074996615625420e+10 6.8399452804377966e+11 -1.2730861976538324e+08 2.7359996580000423e+10 8.2079343365253589e+11 6.3654303672968522e+06 2.9639996295000458e+10 1.9151846785225828e+12 -1.0184686034752984e+08 2.9924996259375465e+10 1.1947308948674746e+12 -3.8181422587255287e+08 5.4149993231250839e+10 2.6675786593707410e+13 -4.9650360873402004e+09 Processor 1 Mat Object: 1 MPI processes type: seqdense 5.4149993231250839e+10 1.9183199555758315e+12 5.9880080526417112e+08 7.8374990203126236e+10 5.7441876402563477e+12 1.7367770705944958e+09 8.0939989882501266e+10 5.7347670250898613e+12 1.7900992494213123e+09 8.1509989811251282e+10 5.7751527083651416e+12 1.8027055821637158e+09 8.4074989490626328e+10 6.1543675577970762e+12 1.8557943339637902e+09 1.0829998646250171e+11 9.5915997778791719e+12 2.9940040263208551e+09 ==== end of submatrix with ISs created with PETSC_COMM_WORLD ====== ======== submatrix with ISs created with PETSC_COMM_SELF ========= Processor 0 Mat Object: 1 MPI processes type: seqdense 2.7074996615625420e+10 6.8399452804377966e+11 -1.2730861976538324e+08 2.7074996615625420e+10 6.8399452804377966e+11 -1.2730861976538324e+08 2.7359996580000423e+10 8.2079343365253589e+11 6.3654303672968522e+06 2.9639996295000458e+10 1.9151846785225828e+12 -1.0184686034752984e+08 2.9924996259375465e+10 1.1947308948674746e+12 -3.8181422587255287e+08 5.4149993231250839e+10 2.6675786593707410e+13 -4.9650360873402004e+09 Processor 1 Mat Object: 1 MPI processes type: seqdense 5.4149993231250839e+10 1.9183199555758315e+12 5.9880080526417112e+08 7.8374990203126236e+10 5.7441876402563477e+12 1.7367770705944958e+09 8.0939989882501266e+10 5.7347670250898613e+12 1.7900992494213123e+09 8.1509989811251282e+10 5.7751527083651416e+12 1.8027055821637158e+09 8.4074989490626328e+10 6.1543675577970762e+12 1.8557943339637902e+09 1.0829998646250171e+11 9.5915997778791719e+12 2.9940040263208551e+09 ==== end of submatrix with ISs created with PETSC_COMM_SELF ====== The results are identical. I really agree with you that only sequential ISs should be used when creating submatrix by MatGetSubMatrices(). The above results may be just a coincidence. I take your suggestions and I am using only sequential ISs with MatGetSubMatrices() now. Yet another question: Can I use IS which have duplicate entries in it? The documentation of MatGetSubMatrices() says that "The index sets may not have duplicate entries", so I think no duplicated entries are allowed in the IS. But again, I just tried IS which have duplicated entries. And the resultant submatrices for each processor seemd to be correct. In fact, in the above sample submatrices I showed, the submatrix owned by Processor 0 has its row IS specified as [x, x, a, b, c, d]. Where 'x' means the duplicated entries in the IS. Then I got two identical rows in the submatrix owned by Processor 0. But I think I was doing this incorrectly. Thanks. HU Yaoyu > > Yaoyu: > > "MatGetSubMatrices() can extract ONLY sequential submatrices > (from both sequential and parallel matrices). Use MatGetSubMatrix() > to extract a parallel submatrix." > > Using parallel IS for MatGetSubMatrices() definitely incorrect, unless you > only use one process. > >> >> And further. I checked the submatrices I obtained by >> MatGetSubMatrices() with both parallel ISs and sequential ISs. The >> submatrices are identical. Is it means that I could actually use >> parallel IS (created with PETSC_COMM_WORLD) ? >> > > What did you compare with? I do not understand what submatrices would > obtain with parallel IS using more than one process. > > Hong > >> >> > Yaoyu : >> > MatGetSubMatrices() returns sequential matrices. >> > IS must be sequential, created with PETSC_COMM_SELF. >> > See petsc/src/mat/examples/tests/ex42.c >> > >> > Check your submatrices. >> > >> > Hong >> > >> > Hi everyone, >> >> >> >> I am trying to get values owned by other processors of a parallel >> matrix. >> >> >> >> I tried to create a sub-matrix by using MatGetSubMatrices(), and then >> >> MatGetRow() on the sub-matrix. But MatGetRow() give me the following >> >> error message: >> >> >> >> ===== Error message begins ===== >> >> >> >> No support for this operation for this object type >> >> only local rows >> >> >> >> ===== Error message ends ===== >> >> >> >> The parallel matrix is a parallel dense matrix. The ISs for >> >> MatGetSubMatrices() are created using ISCreateGeneral() and >> >> PETSC_COMM_WORLD. The row IS is sorted by ISSort(). >> >> >> >> What did I mistake while using the above functions? Is there a better >> >> way to get access to matrix values owned by other processor? >> >> >> >> Thanks! >> >> >> >> HU Yaoyu >> >> >> > -------------- next part -------------- >> > An HTML attachment was scrubbed... >> > URL: < >> http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160413/6be45909/attachment.html >> > >> > >> > ------------------------------ >> > >> > _______________________________________________ >> > petsc-users mailing list >> > [email protected] >> > https://lists.mcs.anl.gov/mailman/listinfo/petsc-users >> > >> > >> > End of petsc-users Digest, Vol 88, Issue 41 >> > ******************************************* >>
