On Dec 29, 2011, at 7:08 AM, Jed Brown wrote:
> On Thu, Dec 29, 2011 at 01:27, Rahul Praghanmor <praghanmor at gmail.com>
> wrote:
> I successfully implement PETSc to solve on a single zone unstructured mesh
> i.e. serial implementation has done and tested.
> Now I want to implement a PETSc on a multi-zone mesh.The mesh partitioning in
> multi-zone is done using a separate tool, Metis. The multi-zone mesh is read
> in solver.I don't understand how to do the global indexing for multi-zone
> mesh to form a global matrix required for PETSc.
>
> Start with a non-overlapping partition (usually what is provided by Metis).
> Distribute the mesh according to this partition, then compute the local size
> and use MPI_Scan() to compute the offset of your piece in the global
> ordering. You can learn the global index of ghosted points by sending it from
> owner to ghoster through the overlap.
You can optionally use the PetscLayout object (I've included the manual page
for PetscLayoutCreate() below since I just improved it) to avoid having to use
MPI_Scan() directly. You just set the local sizes based on your new number of
grid points per MPI process and use PetscLayoutSetUp() and then you can query
the global starting point (rstart) etc of each process using the other routines
mentioned in the manual page.
PetscLayoutCreate - Allocates PetscLayout space and sets the map contents
to the default.
Collective on MPI_Comm
Input Parameters:
+ comm - the MPI communicator
- map - pointer to the map
Level: developer
Notes: Typical calling sequence
PetscLayoutCreate(MPI_Comm,PetscLayout *);
PetscLayoutSetBlockSize(PetscLayout,1);
PetscLayoutSetSize(PetscLayout,n) or
PetscLayoutSetLocalSize(PetscLayout,N);
PetscLayoutSetUp(PetscLayout);
Optionally use any of the following:
PetscLayoutGetSize(PetscLayout,PetscInt *); or
PetscLayoutGetLocalSize(PetscLayout,PetscInt *;)
PetscLayoutGetRange(PetscLayout,PetscInt *rstart,PetscInt *rend); or
PetscLayoutGetRanges(PetscLayout,const PetscInt *range[])
PetscLayoutDestroy(PetscLayout);
The PetscLayout object and methods are intended to be used in the PETSc
Vec and Mat implementions; it is often not needed in
user codes unless you really gain something in their use.
Fortran Notes:
Not available from Fortran
.seealso: PetscLayoutSetLocalSize(), PetscLayoutSetSize(),
PetscLayoutGetSize(), PetscLayoutGetLocalSize(), PetscLayout,
PetscLayoutDestroy(),
PetscLayoutGetRange(), PetscLayoutGetRanges(),
PetscLayoutSetBlockSize(), PetscLayoutGetBlockSize(), PetscLayoutSetUp()
