Mark Adams <[email protected]> writes: > On Tue, Jun 26, 2018 at 1:06 PM Satish Balay <[email protected]> wrote: > >> I wonder if these jobs are scheduled in such a way so that they are not >> oversubscribed. >> >> i.e number_mpi_jobs_per_node * number_of_openmp_threads_per_node <= >> no_of_cores_per_node > > > You are right(ish). This fixed the problem: > > export OMP_NUM_THREADS=1
This is a frequent cause of performance woes. I think PETSc should do some checking for oversubscription, perhaps using omp_num_threads() and MPI_Comm_split_type(MPI_COMM_TYPE_SHARED) along with https://stackoverflow.com/questions/150355/programmatically-find-the-number-of-cores-on-a-machine#150971 Such oversubscription could be reported prominently in -log_view.
