> > Is the scatter created with VecScatterCreateToAll()? If so, internally > the VecScatterBegin/End will use VecScatterBegin_MPI_ToAll() which then uses > a MPI_Allgatherv() to do the communication. You can check in the debugger > for this (on 2 processes) by just putting a break point in > VecScatterBegin_MPI_ToAll() to confirm if it is called.
Alas, not I did not use VecScatterCreateToAll and VecScatterCreateToAll will take some code changes. There are boundary conditions in the destination vector, and so we scatter into a larger vector the the global size of the PETSc vector, using a general IS. With code that looks like this: call ISCreateGeneral(PETSC_COMM_SELF,nreal,petsc_xgc,PETSC_COPY_VALUES,is,ierr) call VecScatterCreate(this%xVec,PETSC_NULL_OBJECT,vec,is,this%from_petsc,ierr) ! reverse scatter object If we want to make this change then I could help a developer or you can get me set up with a (small) test problem and a branch and I can do it at NERSC. Thanks,
