On Wed, Dec 30, 2020 at 6:47 PM Jed Brown <j...@jedbrown.org> wrote: > Mark Adams <mfad...@lbl.gov> writes: > > > I see that ASM has a DM and can get subdomains from it. I have a DMForest > > and I would like an ASM that has a subdomain for each field. How might I > go > > about doing this? (the fields are not coupled in the matrix so this would > > give a block diagonal matrix, and thus exact with LU sub solvers. > > The fields are already not coupled or you want to filter the matrix and > give back a single matrix with coupling removed? >
They are not coupled. It is is a block diagonal matrix but not stored that way by Plex. > > You can use Fieldsplit to get the math of field-based block Jacobi (or > ASM, but overlap with fields tends to be expensive). Neither FieldSplit or > ASM can run the (additive) solves concurrently (and most libraries would > need something to drive the threads). > No overlap, MPI serial. Very simple. I just want a direct solver and I want to exploit the 10-way parallelism that is just sitting there. And I see that PCApply_ASM would need some very particular non-blocking semantics in KSPSolve and VecScatter to work so I would assume that a new driver would be required or just rearrange the current one a bit if you don't mind losing a bit of cache reuse in doing each block all at once. > > > I am then going to want to get these separate solves to be run in > parallel > > on a GPU (I'm talking with Sherry about getting SuperLU working on these > > small problems). In looking at PCApply_ASM it looks like this will take > > some thought. KSPSolve would need to be non-blocking, etc., or a new > apply > > op might be needed. > > > > Thanks, > > Mark >
