On Tue, Apr 3, 2018 at 10:31 AM Satish Balay <ba...@mcs.anl.gov> wrote:
> On Tue, 3 Apr 2018, Derek Gaston wrote: > > Which does bring up a point: I have been able to do solves before with > > ~50,000 separate PETSc solves without issue. Is it because I was working > > with MVAPICH on a cluster? Does it just have a higher limit? > > Don't know - but thats easy to find out with a simple test code.. > I get 2044 using mvapich on my cluster too. The only thing I can think of as to why those massive problems work for me is that any one MPI process is not involved in more than ~2000 processors (because the communicators are split as you go down the hierarchy). At most, a single MPI process will see ~hundreds of PETSc solves but not thousands. That said: it's just because of the current nature of the solves I'm doing - it's definitely possible to have that not be the case with MOOSE. Derek