Exactly! My tests were performed on Kraken (Cray XT5) and a Cray XE6 and I was using iterative solvers on the coarse grid.
On 22 November 2013 00:10, Jed Brown <[email protected]> wrote: > Dave May <[email protected]> writes: > > > I argue it does matter as I've seen runs on 32k cores where a huge amount > > of time is spent in those global reductions. I can provide an > > implementation which uses a sub comm (PCSemiRedundant) if someone thinks > > doing reductions on less cores is beneficial. > > It doesn't matter much on Blue Gene, but is a big deal on older Crays. > Aires seems to be in between. The default GAMG configuration doesn't do > any reductions in the coarse grid, so the issue is moot. If an > iterative coarse solver was used, I think we would be more motivated to > put the coarse problem on a subcomm. >
