On Dec 31, 2013, at 2:37 AM, Jed Brown <[email protected]> wrote:
> How would you propose to handle Look, what I have now is basically like a re-implementation of a VecScatter between two distributed vectors. It's only partly about coming up with cute new algorithms under the hood: my foremost claim is about the integrative API. > irregular communication patterns Sparse matrices? Basically a VecScatter. > such as > you would get with a small number of dense rows in a sparse matrix, or > the row/column communicators for dense linear algebra? How are > reductions expressed and implemented? Are the two questions related? Reductions: my theory paper describes it. I have no implementation yet. Small number of dense rows: are you implying that a matrix-vector product with such a matrix should be a combination of regular sparse VecScatter and a reduction? Interesting thought. As a refinement many miles down the line I could think about it. > I think that the setup costs are actually critical. They are not always > amortized in either dynamic execution models or methods with frequent > adaptivity, particle transport, self-contact, etc. The MPI model may > already have two-sided knowledge about the new communication pattern or > know a good way to use collectives. It's more work for the programmer > to specify both sides of the communication, but dynamic exchange is > significantly more expensive. I'm not really sure what you're arguing here, but I'll offer the following. 1. any setup algorithm you have in Petsc I can straightaway copy to my software 2. adaptivity: rather than doing a full setup you could do an update of an existing one. Since adaptive refinement or load balancing or such means a change of one of the distributions involved, this is a formal transformation just like the alpha-beta transform. Therefore the system can calculate the effect of it. I could implement it by composition (as in: execute the one after the other) of two transformations. And then I could think about a transformation of transformations that collapses them into one, which would be a local update of the communication setup. This is also several miles down the line. Thanks for your thoughts. If I could get someone to fund this stuff me and a couple of students/post-docs could have no ends of fun turning this into useful software. Victor. -- Victor Eijkhout, 512 471 5809 (w) Texas Advanced Computing Center, The University of Texas at Austin
