Hmm, sometimes you talk about MatMPIAIJSetPreallocation() and sometimes
MatMPISBAIJSetPreallocation() which one is it?
If a process has zero rows that should not affect the nonew variable.
It is crucial the the SetPreallocation routines be called on ALL processes
that share the matrix, even if they have zero rows, but it shouldn’t matter
what the values are for a matrix with zero rows.
Can you send use your test case so we can track down the problem?
Barry
On Jan 15, 2014, at 5:57 AM, Projet_TRIOU <[email protected]> wrote:
> Hi all,
>
> I switched from PETSc 3.3 to PETSc 3.4.3 and all was fine except for
> one of my test case on 3 processors where one processor was
> dealing an empty local part of the global matrix.
>
> My code hangs just during the call at MatAssemblyEnd:
>
> ierr = MatMPIAIJSetPreallocation(MatricePetsc, PETSC_DEFAULT, d_nnz.addr(),
> PETSC_DEFAULT, o_nnz.addr());
> ...
> ierr = MatAssemblyEnd(MatricePetsc, MAT_FINAL_ASSEMBLY);
>
> When I debugged, I notice on the empty processor, that in
> src/mat/impls/aij/mpi/mpiaij.c:
>
> if (!((Mat_SeqAIJ*)aij->B->data)->nonew) {
> ierr =
> MPI_Allreduce(&mat->was_assembled,&other_disassembled,1,MPIU_BOOL,MPI_PROD,PetscObjectComm((PetscObject)mat));CHKERRQ(ierr);
> if (mat->was_assembled && !other_disassembled) {
> ierr = MatDisAssemble_MPIAIJ(mat);CHKERRQ(ierr);
> }
>
> ((Mat_SeqAIJ*)aij->B->data)->nonew was 0 on the "empty" processor
> and -2 on the 2 others...
>
> I bypassed my problem with a different call to MatMPIAIJSetPreallocation():
>
> if (nb_rows==0) // Local matrix is empty
> ierr = MatMPISBAIJSetPreallocation(MatricePetsc, block_size_, 0,
> NULL, 0, NULL);
> else
> ierr = MatMPISBAIJSetPreallocation(MatricePetsc, block_size_,
> PETSC_DEFAULT, d_nnz.addr(), PETSC_DEFAULT, o_nnz.addr());
>
> Now, it runs well. So I don't know if it is a PETSc regression or if I was
> abusively
> calling MatMPISBAIJSetPreallocation with d_nnz/o_nnz empty arrays....
>
> Thanks,
>
> Pierre
>
