You cannot just split a 2d array among some processes and a 3d array across the same processes and expect somehow that the layouts of two of the indices will match between the two arrays.
You can run, for example with -da_view to see how they two arrays are being laid out in parallel by default. You will need set certain values in the m,n, and p arguments to DMDACreate2d() and 3d to get layouts that match. Note that these are not the M,N, and P arguments. Barry On Aug 27, 2011, at 4:09 PM, Likun Tan wrote: > the size of temp and H is 9*9*201, the size of S is 9*9. > > I used 2 processors, the returned values are > temp: 0 0 0 9 9 101 > temp: 0 0 101 9 9 100 > H: 0 0 101 9 9 100 > S: 0 5 101 9 4 100 > > I am not sure what you mean by how DAs were constructed, the command i > used for construction is: > DACreate2d(PETSC_COMM_WORLD, DA_NONPERIODIC, DA_STENCIL_STAR, 9, 9, > PETSC_DECIDE, PETSC_DECIDE, 1, 0, PETSC_NULL, PETSC_NULL, &da2D); > DACreate3d(PETSC_COMM_WORLD, DA_NONPERIODIC, DA_STENCIL_STAR, 9, 9, 201, > PETSC_DECIDE, PETSC_DECIDE, PETSC_DECIDE, 1, 0, PETSC_NULL, PETSC_NULL, > PETSC_NULL, &da3D); > > i got some sense of the problem after watching m, n, p, but i don't know > how to avoid it. > > On Sat, August 27, 2011 4:53 pm, Jed Brown wrote: >> On Sat, Aug 27, 2011 at 15:49, Likun Tan <likunt at andrew.cmu.edu> wrote: >> >> >>> I attached my code again, p was used in defining temp and H, when >>> defining S, we don't need p and zs: >>> >>> >> >> You need p to know the limits of the integral. You need that the domain >> not be decomposed in the z direction if you are going to integrate without >> doing communication. You still haven't shown us how the DAs were >> constructed or what the returned values for (xs,ys,zs,m,n,p) were when got >> them from da3D and what (xs,ys,m,n) were when you got them from da2D. >> > > > >
