Hi guys, I just did something by pure guessing which seems to work and I want to make sure its the right thing!
I have a specific layout for my vectors that look like this -------------------------------------------------------------- | ghost values | local values | ghost values | -------------------------------------------------------------- 0, 1, 2, ... m, m+1, ... m+n, m+n+1 ... N which means all nodes with index [0,m) and [m+n, N) are to be treated as ghost and all intermediate ones as local. Since PETSc stores the ghost values at the end of ghosted vec, so far I have been manually mapping back and forth between petsc and my application numbering. After profiling my code, it turned out that about 15% of run-time was just doing this mapping which is absolutely ridiculous! Anyway, to fix this now I set up an ISLocalToGlobalMapping object using my own application original local2global relation shown above. Then I create the petsc vec like this: // Create PetscVec // We do not care about the actual global index of ghost nodes at this point std::vector<PetscInt> ghost_nodes(N - n -1, 0); ierr = VecCreateGhost(mpicomm, n+1, num_global, ghost_nodes.size(), (const PetscInt*)&ghost_nodes[0], &v); CHKERRQ(ierr); // Apply mapping ierr = VecSetLocalToGlobalMapping(v, mapping); CHKERRQ(ierr); After this point I do the usual VecGetArray on the vec and set the values, but this time without any extra mapping ... my very simple test seems to be ok. Is this the correct way of using ISLocalToGlobalMapping? I guess I'm suspicious because VecCreateGhost is supposed to internally set the mapping and it is supposed to position ghost nodes at the end of the array which I don't want it to do... Thanks and sorry about the long email!