Hello,
I create a MATAIJ on 4 processors using local sizes
(0) Set matrix A to local size 2 x 5
(1) Set matrix A to local size 2 x 2
(2) Set matrix A to local size 2 x 2
(3) Set matrix A to local size 2 x 2
which results in a global size of 8 x 11.
Afaik the local size given is not relevant for the parallel layout, so each
processors holds 2 rows x 11 cols. Accordingly I can set these using
MatSetValue and alike
But it seems to be important when I do preallocation, i.e. work on the local
submatrices:
(0) Local Submatrix Rows = 0 / 2, Local Submatrix Cols = 0 / 5
(1) Local Submatrix Rows = 2 / 4, Local Submatrix Cols = 5 / 7
(2) Local Submatrix Rows = 4 / 6, Local Submatrix Cols = 7 / 9
(3) Local Submatrix Rows = 6 / 8, Local Submatrix Cols = 9 / 11
(retrieved with MatGetOwnershipRange / MatGetOwnershipRangeColumn)
therefore, if I do a preallocation using MatMPIAIJSetPreallocation I get errors
like
[1]PETSC ERROR: MatSeqAIJSetPreallocation_SeqAIJ() line 3567 in
/home/florian/software/petsc/src/mat/impls/aij/seq/aij.c
nnz cannot be greater than row length: local row 0 value 4 rowlength 2
Easiest way to work around that would be to set the number of local cols (here
always 5/2/2/2) to 11. Setting the number of global columns to 11 would
probably have the same effect.
Problem is that each processor only knows the number of local columns, so I
would need some communication for that.
Another possibility that comes to my mind is to declare the Mat only after the
preallocation has been determined and during preallocation the longest line has
been computed.
Are my assumptions correct so far?
How would you cope with that?
Best Regards,
Florian
