Sorry to say, some recent commit has broken the trunk: rhc@bend002 examples]$ mpirun -n 3 ./hello_c [bend001:22289] *** Process received signal *** [bend001:22289] Signal: Segmentation fault (11) [bend001:22289] Signal code: Invalid permissions (2) [bend001:22289] Failing at address: 0x7f354daaa000 [bend001:22290] *** Process received signal *** [bend001:22290] Signal: Segmentation fault (11) [bend001:22290] Signal code: Invalid permissions (2) [bend001:22290] Failing at address: 0x7fa819d81000 [bend001:22289] [ 0] /lib64/libpthread.so.0[0x38e320f710] [bend001:22289] [ 1] /lib64/libc.so.6[0x38e26845ad] [bend001:22289] [ 2] /home/common/openmpi/build/svn-trunk/lib/openmpi/mca_btl_vader.so(+0x3b0b)[0x7f3549924b0b] [bend001:22289] [ 3] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(mca_btl_base_select+0x1cc)[0x7f354db62a21] [bend001:22289] [ 4] /home/common/openmpi/build/svn-trunk/lib/openmpi/mca_bml_r2.so(mca_bml_r2_component_init+0x27)[0x7f354a1cfc2c] [bend001:22289] [ 5] [bend001:22290] [ 0] /lib64/libpthread.so.0[0x38e320f710] [bend001:22290] [ 1] /lib64/libc.so.6[0x38e26845ad] [bend001:22290] [ 2] /home/common/openmpi/build/svn-trunk/lib/openmpi/mca_btl_vader.so(+0x3b0b)[0x7fa815bfbb0b] [bend001:22290] [ 3] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(mca_bml_base_init+0xe2)[0x7f354db6189e] [bend001:22289] [ 6] /home/common/openmpi/build/svn-trunk/lib/openmpi/mca_pml_ob1.so(+0x7cc3)[0x7f35492c3cc3] [bend001:22289] [ 7] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(mca_pml_base_select+0x29c)[0x7f354db88261] [bend001:22289] [ 8] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(ompi_mpi_init+0x685)[0x7f354dafbc7b] [bend001:22289] [ 9] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(mca_btl_base_select+0x1cc)[0x7fa819e39a21] [bend001:22290] [ 4] /home/common/openmpi/build/svn-trunk/lib/openmpi/mca_bml_r2.so(mca_bml_r2_component_init+0x27)[0x7fa8164a6c2c] [bend001:22290] [ 5] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(mca_bml_base_init+0xe2)[0x7fa819e3889e] [bend001:22290] [ 6] /home/common/openmpi/build/svn-trunk/lib/openmpi/mca_pml_ob1.so(+0x7cc3)[0x7fa81559acc3] [bend001:22290] [ 7] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(mca_pml_base_select+0x29c)[0x7fa819e5f261] [bend001:22290] [ 8] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(MPI_Init+0x185)[0x7f354db2f156] [bend001:22289] [10] ./hello_c[0x400806] [bend001:22289] [11] /lib64/libc.so.6(__libc_start_main+0xfd)[0x38e261ed1d] [bend001:22289] [12] ./hello_c[0x400719] [bend001:22289] *** End of error message *** /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(ompi_mpi_init+0x685)[0x7fa819dd2c7b] [bend001:22290] [ 9] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(MPI_Init+0x185)[0x7fa819e06156] [bend001:22290] [10] ./hello_c[0x400806] [bend001:22290] [11] /lib64/libc.so.6(__libc_start_main+0xfd)[0x38e261ed1d] [bend001:22290] [12] ./hello_c[0x400719] [bend001:22290] *** End of error message *** [bend001:22291] *** Process received signal *** [bend001:22291] Signal: Segmentation fault (11) [bend001:22291] Signal code: Invalid permissions (2) [bend001:22291] Failing at address: 0x7f498fc96000 [bend001:22291] [ 0] /lib64/libpthread.so.0[0x38e320f710] [bend001:22291] [ 1] /lib64/libc.so.6[0x38e26845ad] [bend001:22291] [ 2] /home/common/openmpi/build/svn-trunk/lib/openmpi/mca_btl_vader.so(+0x3b0b)[0x7f498795db0b] [bend001:22291] [ 3] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(mca_btl_base_select+0x1cc)[0x7f498fd4ea21] [bend001:22291] [ 4] /home/common/openmpi/build/svn-trunk/lib/openmpi/mca_bml_r2.so(mca_bml_r2_component_init+0x27)[0x7f498c3bbc2c] [bend001:22291] [ 5] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(mca_bml_base_init+0xe2)[0x7f498fd4d89e] [bend001:22291] [ 6] /home/common/openmpi/build/svn-trunk/lib/openmpi/mca_pml_ob1.so(+0x7cc3)[0x7f49872fccc3] [bend001:22291] [ 7] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(mca_pml_base_select+0x29c)[0x7f498fd74261] [bend001:22291] [ 8] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(ompi_mpi_init+0x685)[0x7f498fce7c7b] [bend001:22291] [ 9] /home/common/openmpi/build/svn-trunk/lib/libmpi.so.0(MPI_Init+0x185)[0x7f498fd1b156] [bend001:22291] [10] ./hello_c[0x400806] [bend001:22291] [11] /lib64/libc.so.6(__libc_start_main+0xfd)[0x38e261ed1d] [bend001:22291] [12] ./hello_c[0x400719] [bend001:22291] *** End of error message *** -------------------------------------------------------------------------- mpirun noticed that process rank 0 with PID 22289 on node bend001 exited on signal 11 (Segmentation fault). -------------------------------------------------------------------------- 3 total processes killed (some possibly by mpirun during cleanup) [rhc@bend002 examples]$
Nathan: can you please take a look? Ralph
