On Thu, Dec 13, 2012 at 7:41 AM, Thomas Witkowski <thomas.witkowski at tu-dresden.de> wrote: > I have some problem in the implementation of my multilevel FETI DP code, > where two block structured matrices must be multiplied. I'll give my best to > explain the problem, may be one of you have an idea how to implement it. I > think, the best is to make a small example: lets assume we have 16 > subdomains, uniformly subdividing a unit square. Each of the subdomain > matrices is purely local, thus they have the communicator PETSC_COMM_SELF. > Each of them is of size n x n. There is a coarse grid matrix, with > communicator PETSC_COMM_WORLD and of size m x m. The coupling matrices > between the global coarse grid and the local matrices are also global, so > they are of size 16n x m and m x 16n, respectively. So far, everything is > fine and works perfectly. Now I introduce four "local coarse grids", each of > them couples four local subdomains, and is defined on a subset communicator > of PETSC_COMM_WORLD. Say, each "local coarse grid" matrix is of size p x p, > and there are also coupling matrices of size 4n x p and p x 4n. Now I have > to perform a MatMatMult of the local coarse coupling matrices p x 4n with > the global coupling matrix 16n x m. So the final matrix is of size 4p x m. > But I cannot perform the MatMatMult, as the matrix sizes do not fit and the > communicators are not compatible. > > Is it possible to understand, what I want to do? :) Any idea, how to > implement it?
It sounds like you need to redistribute the matrix before the MatMatMult. I think you can do this with MatGetSubmatrix(), if I understand your problem correctly. You probably need to move the matrix from the subcomm to the global comm first, with empty entries on some procs. I would just do it the naive way first, then profile to see how it does. Matt > Thomas -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
