[OMPI devel] Barrier() after Finalize() when a file handle is leaked.

[Lisandro Dalcin]

I've tested this with (--enable-debug --enable-picky
--enable-mem-debug) 1.4.2 and 1.5rc6. Despite being debug builds, a
mpi4py user got the same with (likely release) builds in both Ubuntu
and OS X.

$ cat open.c
#include <mpi.h>
int main(int argc, char *argv[]) {
  MPI_File f;
  MPI_Init(&argc, &argv);
  MPI_File_open(MPI_COMM_WORLD, "test.plt", MPI_MODE_RDONLY, MPI_INFO_NULL, &f);
  /* MPI_File_close(&f); */
  MPI_Finalize();
  return 0;
}

$ mpicc open.c

$ ./a.out
*** The MPI_Barrier() function was called after MPI_FINALIZE was invoked.
*** This is disallowed by the MPI standard.
*** Your MPI job will now abort.
[trantor:15145] Abort after MPI_FINALIZE completed successfully; not
able to guarantee that all other processes were killed!


So if you open a file but never close it, a MPI_Barrier() gets called
after MPI_Finalize(). Could that come from a finalizer ROMIO callback?
However, I do not get this failure with MPICH2, and Open MPI seems to
behave just fine regarding MPI_Finalized(), the code below work as
expected:

#include <mpi.h>
#include <stdio.h>

static int atexitmpi(MPI_Comm comm, int k, void *v, void *xs) {
  int flag;
  MPI_Finalized(&flag);
  printf("atexitmpi: finalized=%d\n", flag);
  MPI_Barrier(MPI_COMM_WORLD);
}

int main(int argc, char *argv[]) {
  int keyval = MPI_KEYVAL_INVALID;
  MPI_Init(&argc, &argv);
  MPI_Comm_create_keyval(MPI_COMM_NULL_COPY_FN, atexitmpi, &keyval, 0);
  MPI_Comm_set_attr(MPI_COMM_SELF, keyval, 0);
  MPI_Finalize();
  return 0;
}



-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169