What about recycling of Krylov subspaces and improved support for initial guesses (already there, maybe we can add other low-order methods)?
On Jul 2, 2016, at 10:44 PM, Hong <[email protected]> wrote: > Efficient and scalable MatGetSubmatrix() and assemble submatrices into a > matrix -- used for multiphysic simulation, e.g., wash project we are doing > now. > > Hong > > On Sat, Jul 2, 2016 at 2:46 AM, Patrick Sanan <[email protected]> wrote: > Maybe a general class of opportunities for PETSc could be wrapped up > under "good abstractions for memory space awareness". That is, all > these topics have been discussed recently: > > 1. Re Jeff's excellent suggestion about OpenMP, it would be nice if > more were done to promote the alternative, MPI-based shared memory > capability. It's quite clear that this is a good way to go, but this > doesn't mean that people (who often equate shared memory parallelism > with threads) will use it, so the more that can be done to make the > internals of the library use this approach and the more that the > classes (DM in particular) can make it easy to do things "the right > way", the better. > > 2. As we saw from Richard's talk on KNL, that device will feature > (when used a certain way) one NUMA domain which can provide much > greater memory bandwidth. As in the discussions here before on the > topic, it's a real challenge to figure out how to to make something > like this usable via PETSc, and an even greater one to pick defaults > in a way that won't sometimes produce bewildering performance results. > Nevertheless, there's a good chance this is the kind of hardware that > people will be running PETSc on, and given that this is something > which attacks memory bandwidth bottlenecks, something that could speed > up a lot of PETSc code. > > 3. People will likely also be running on machines with > coprocessors/accelerators/etc for a while, and there needs to be > plumbing to deal with the fact that each device may provide an extra > memory space which needs to be managed properly in the flat MPI > environment. This is of course related to point 1, as good use of > MPI-3 shared memory seems like a reasonable way forward. > > 4. Related topics re MPI-4/endpoints and how PETSc really could be > used properly from an existing multi-threaded environment. Karl has > talked about the challenges of doing this the right way a lot > recently. > > With all of these, introducing some clever abstractions to allow these > sorts of things to be as transparent/automatic/encapsulated as > possible might be very valuable and well-used additions to the > library. > > On Sat, Jul 2, 2016 at 1:13 AM, Jeff Hammond <[email protected]> wrote: > > Obviously, holistic support for OpenMP is critical to the future of PETSc > > :-D > > > > On a more serious note, Matt and I have discussed the use of PETSc for > > sparse multidimensional array computations for dimensions greater than 2, > > also known as tensor computations. The associated paper describing previous > > work with dense arrays is > > http://www.eecs.berkeley.edu/Pubs/TechRpts/2012/EECS-2012-210.html. There > > was even an unsuccessful SciDAC application proposal that described how > > PETSc could be used for that domain when sparsity is important. To start, > > all we'd need is sparse matrix x sparse matrix multiplication, which I hear > > the multigrid folks also need. Sparse times dense is also important. > > Sparse tensor factorization would also help, but I get that there are enough > > open math questions there that it might be impractical to try to implement > > something in PETSc in the near future. > > > > Maybe I am just biased because I spend all of my time reading > > www.nextplatform.com, but I hear machine learning is becoming an important > > HPC workload. While the most hyped efforts related to running inaccurate - > > the technical term is half-precision - dense matrix multiplication as fast > > as possible, I suspect that more elegant approaches will prevail. > > Presumably there is something that PETSc can do to enable machine learning > > algorithms. As most of the existing approaches use silly programming models > > based on MapReduce, it can't be too hard for PETSc to do better. > > > > Jeff > > > > On Fri, Jul 1, 2016 at 2:32 PM, Barry Smith <[email protected]> wrote: > >> > >> > >> The DOE SciDAC institutes have supported PETSc linear solver > >> research/code development for the past fifteen years. > >> > >> This email is to solicit ideas for linear solver research/code > >> development work for the next round of SciDAC institutes (which will be a 4 > >> year period) in PETSc. Please send me any ideas, no matter how crazy, on > >> things you feel are missing, broken, or incomplete in PETSc with regard to > >> linear solvers that we should propose to work on. In particular, issues > >> coming from particular classes of applications would be good. Generic > >> "multi > >> physics" coupling types of things are too general (and old :-)) while work > >> for extreme large scale is also out since that is covered under another > >> call > >> (ECP). But particular types of optimizations etc for existing or new codes > >> could be in, just not for the very large scale. > >> > >> Rough ideas and pointers to publications are all useful. There is an > >> extremely short fuse so the sooner the better, > >> > >> Thanks > >> > >> Barry > >> > >> > >> > > > > > > > > -- > > Jeff Hammond > > [email protected] > > http://jeffhammond.github.io/ >
