On Wed, 10 Mar 2010 22:25:43 -0500, Gus Correa <g...@ldeo.columbia.edu> wrote:
> Ocean dynamics equations, at least in the codes I've seen, > normally use "pencil" decomposition, and are probably harder to > handle using 3D "chunk" decomposition (due to the asymmetry imposed by > gravity). There is also a lot to be said for the strength of coupling. Ocean codes do "tridiagonal solves" in columns, and these would no longer be trivially cheap (in fact the structure of the code would need to change) if the partition also broke up the vertical. Since the domain is so anisotropic and the coupling (at least aside from the barotropic mode) is so much stronger in the vertical than the horizontal, it is good to decompose with columns always kept together. In a fully implicit code, these column solves would quit being mandatory, but the availability of a "line smoother" for multigrid still favors this type of decomposition. Also note that in domain decomposition algorithms (like additive Schwarz, and balancing Neumann-Neumann), the asymptotics scale with the maximum number of subdomains required to cross the domain, and/or with the number of elements along the longest edge of a subdomain. This tends to favor partitioning in 3D, unless the physics/domain is sufficiently anisotropic to overcome this preference. Also depending on Derek's application, he may want to use a library like PETSc to handle the decomposition and updates. Certainly this is true if the application may ever need solvers; in my opinion, it is also true unless this is a toy problem being used to learn MPI. If you really want to write this stuff yourself, it's still worth looking at the discussion in PETSc user's manual. Jed
