On Mon, Mar 7, 2011 at 11:32 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> > On Mar 6, 2011, at 11:03 PM, Jack Poulson wrote: > > > Sorry to hit the list so quickly again, but the PCShellSetApply > documentation doesn't say anything about how the 'xin' and 'xout' vectors > will be distributed, or if it's up to the user to determine it. > > > > Given that a distributed Mat implies a right and left vector > distribution, I would assume that the 'xin' should be distributed like the > 'left' vector distribution for a left preconditioner. > > > > Is it up to the application routine to determine the 'xout' vector > distribution? If so, is there a performance penalty for making it something > other than the 'right' vector distribution implied by the matrix registered > with the KSP? If not, what distribution does it need to be? > > The PCShell doesn't determine the vector layouts at all. That is > determined by the KSP that is using the PC. So basically your PC should > handle any distribution; if your PCShell can only handle certain > distributions then you need to make sure that the Vecs and Mats you provide > to KSP or SNES (or TS) match what the PCShell needs. > > Barry > > It turns out that I had simply misunderstood the way PETSc was distributing matrices under the hood. I was assuming that it was best to only distribute the rows of the matrix using something along the lines of MatSetSizes( A, N/nproc, N, N, N ), but Jed informed me that the local size arguments for MatSetSizes only constrain the left and right vector distributions implied by the matrix distribution (through rmap and cmap, respectively). Also, I now know that KSP expects the right and left vector distributions to be the same for square matrices, so I should instead create my matrix along the lines of MatSetSizes( A, N/nproc, N/nproc, N, N ), and then both the in and out vectors for the PCShell application routine should be distributed in the same manner. Thank you both for clearing up my misconceptions. Jack -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20110307/47c4e859/attachment.htm>
