Hi Barry, the order is determined by MPI_Cart_create. I've tried to swap x and y in the DA to make the problem go away but haven't succeeded yet. Not enough brain cells I suspect ... still thinking about it. Thanks for the response, Sean
Barry Smith wrote: > > > Sean, > > I don't completely understand what goes wrong. Is it because YOUR > application orders the processors related to geometry in the following > way? > > ^ y direction > | > 2 5 8 > 1 4 7 > 0 3 6 > > -> x direction > > Or is this something inherent in MPI_Cart_create? > > PETSc does it so > > ^ y direction > | > 6 7 8 > 3 4 5 > 0 1 2 > > -> x direction > > If you want to "force" the PETSc DA to match the first case you > could just > "lie" to PETSc and treat the x direction as the y direction and the x > as the y. > Thus you would flip the i and j indices. You could do the same trick in > 3d. > > There is no easy way to change the DA to do the ordering as the first > case above or support both approaches (possible? yes, but ugly > duplicate code to handle the two cases). > > I've cc:ed to Bill since he would know more about the details of > MPI_cart_create(). > > Barry > > On Mon, 15 May 2006, Sean Dettrick wrote: > >> Hi, >> >> I'm trying to use DACreate2d and KSP in my existing MPI application. >> I already have a Cartesian communicator established, and I set >> PETSC_COMM_WORLD equal to it and then call PetscInitialize. >> >> This works fine on a prime number of CPUs, because there is only one >> possible ordered MPI layout in one dimension. But with a non-prime >> number there are two possible ordered layouts and it just happens >> that my 2D CPU layout (determined by MPI_Cart_create) is the >> transpose of the PETSc 2D CPU layout. Is there a way to organize the >> DA layout more explicitly than with DACreate2d? Or to tell PETSc to >> transpose its CPU order? I also wonder about the 3D case. >> >> thanks >> Sean >> >> > >
