This concerns IBM PE-MPI on iDataPlex, which is likely based upon the cluster 
implementation of PAMI, which is a completely different code base from the PAMI 
Blue Gene implementation.  If you can reproduce it on Blue Gene/Q, I will care.

As an IBM customer, NCAR is endowed with the ability to file bug reports 
directly with IBM related to the products they possess.  There is a link to 
their support system on http://www2.cisl.ucar.edu/resources/yellowstone, which 
is the appropriate channel for users of Yellowstone that have issues with the 
system software installed there.

Jeff

----- Original Message -----
From: "Jed Brown" <[email protected]>
To: "Fande Kong" <[email protected]>, "petsc-users" <[email protected]>
Cc: "Jeff Hammond" <[email protected]>
Sent: Wednesday, June 26, 2013 9:21:48 AM
Subject: Re: [petsc-users] How to understand these error messages

Fande Kong <[email protected]> writes:

> Hi Barry,
>
> If I use the intel mpi, my code can correctly run and can produce some
> correct results. Yes, you are right. The IBM MPI has some bugs.

Fande, please report this issue to the IBM.

Jeff, Fande has a reproducible case where when running on 10k cores and
problem sizes over 100M, this

  MPI_Waitany(2*stash->nrecvs,stash->recv_waits,&i,&recv_status);

returns

      [6724]PETSC ERROR: Negative MPI source: stash->nrecvs=8 i=11
      MPI_SOURCE=-32766 MPI_TAG=-32766 MPI_ERROR=20613892!

It runs correctly for smaller problem sizes, smaller core counts, or for
all sizes when using Intel MPI.  This is on Yellowstone (iDataPlex, 4500
dx360 nodes).  Do you know someone at IBM that should be notified?

-- 
Jeff Hammond
Argonne Leadership Computing Facility
University of Chicago Computation Institute
[email protected] / (630) 252-5381
http://www.linkedin.com/in/jeffhammond
https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond
ALCF docs: http://www.alcf.anl.gov/user-guides

Reply via email to