Samar Khatiwala <[email protected]> writes: > CRAY has finally gotten back with a solution, > which is to use the run-time option -vecscatter_alltoall. Apparently > this is a known issue and according to the HLRN folks passing this > command line option to PETSc seems to work nicely.
This option is good when you have nearly-dense rows or columns (in terms of processors depended on). For problems with actual dense rows or columns, it is good to formulate as a sparse matrix plus a low-rank correction. The other cases are usually poor dof layout, and reordering will make the graph sparser. Sparse problems with good layout usually run faster with the default VecScatter, though there are exceptions (mostly non-PDE problems).
pgpHGSypIegPg.pgp
Description: PGP signature
