On Fri, Jan 20, 2012 at 05:30, <S.H.Jongsma at utwente.nl> wrote: > ** ** > > Hello fellow PETSc users,**** > > ** ** > > While using PETSc I came across some conspicuous behavior regarding the > use of MatTranspose. When I use this function with a MATSEQBAIJ matrix it > performs considerably better than when I use it with a MATMPIBAIJ matrix. I > was wondering if anyone of you noticed the same behavior and if this is the > case, what did you do about it to circumvent the ?problem?. In case no one > came across this problem before, any suggestions that can help solving this > performance issue are welcome. **** > > ** ** > > To make my problem more clear, I will give some details on my > implementation and the test I performed to come to my conclusion. The PETSc > version I use is 3.1-p8. >
Just a friendly reminder to upgrade to petsc-3.2. > I transpose the matrix in place using the following command:**** > > ** ** > > MatTranspose(myMatrix, MAT_REUSE_MATRIX, &myMatrix); > The code is not doing correct preallocation here. Can you try AIJ? It still uses a heuristic, but is more likely to be sufficient. Or, still with MPIBAIJ if you have a square matrix with symmetric nonzero pattern, this out-of-place transpose should perform well. MatDuplicate(myMatrix,MAT_DO_NOT_COPY_VALUES,&myMatrixTranspose); MatTranspose(myMatrix,MAT_REUSE_MATRIX,&myMatrixTranspose); Why do you need a transpose? Sparse transpose is a very expensive operation in parallel (even with correct preallocation) and it should always be avoidable (KSPSolveTranspose(), etc). > **** > > ** ** > > The matrix I used to test the performance is square and consists of 24576 > times 24576 blocks of size 5, which means that the matrix has 122880 rows > (and columns, of course). The number of non-zero blocks in the matrix is: > 291830. Timing the performance gives the following results:**** > > ** ** > > MATSEQBAIJ: 0.320 s**** > > MATMPIBAIJ: 1474 s (running on 1 processor) **** > > MATMPIBAIJ: 376 s (running on 2 processors)**** > > ** ** > > As I said, the difference in performance is quite considerable, so any > suggestion that can help me solving this issue are greatly appreciated. ** > ** > > ** ** > > Thank you in advance,**** > > ** ** > > Kind regards,**** > > ** ** > > ** ** > > Sietse Jongsma**** > > ** ** > > ** ** > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120120/d701d885/attachment.htm>
