I replied a couple days ago (with OMPI users in CC) but got an error last night:
Action: failed Status: 5.0.0 (permanent failure) Diagnostic-Code: smtp; 5.4.7 - Delivery expired (message too old) 'timeout' (delivery attempts: 0) I resent the mail this morning, it looks like it wasn't delivered yet either? I'll try again with another SMTP. Brice Le 10/09/2012 20:37, Douglas Eadline a écrit : > I got no response for this question. Is Open-MX > no longer supported in Open MPI? Or is there someplace else > I should submit this information? I also attached my ompi_info > and omx_info output > > -- > Doug > > >> I built open-mpi 1.6.1 using the open-mx libraries. >> This worked previously and now I get the following >> error. Here is my system: >> >> kernel: 2.6.32-279.5.1.el6.x86_64 >> open-mx: 1.5.2 >> >> BTW, open-mx worked previously with open-mpi and the current >> version works with mpich2 >> >> >> $ mpiexec -np 8 -machinefile machines cpi >> Process 0 on limulus >> FatalError: Failed to lookup peer by addr, driver replied Bad file >> descriptor >> cpi: ../omx_misc.c:89: omx__ioctl_errno_to_return_checked: Assertion `0' >> failed. >> [limulus:04448] *** Process received signal *** >> [limulus:04448] Signal: Aborted (6) >> [limulus:04448] Signal code: (-6) >> [limulus:04448] [ 0] /lib64/libpthread.so.0() [0x3324e0f500] >> [limulus:04448] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x33246328a5] >> [limulus:04448] [ 2] /lib64/libc.so.6(abort+0x175) [0x3324634085] >> [limulus:04448] [ 3] /lib64/libc.so.6() [0x332462ba1e] >> [limulus:04448] [ 4] /lib64/libc.so.6(__assert_perror_fail+0) >> [0x332462bae0] >> [limulus:04448] [ 5] >> /usr/open-mx/lib/libopen-mx.so.0(omx__ioctl_errno_to_return_checked+0x197) >> [0x7fb587418b37] >> [limulus:04448] [ 6] >> /usr/open-mx/lib/libopen-mx.so.0(omx__peer_addr_to_index+0x55) >> [0x7fb58741a5d5] >> [limulus:04448] [ 7] /usr/open-mx/lib/libopen-mx.so.0(+0xdc7a) >> [0x7fb587419c7a] >> [limulus:04448] [ 8] /usr/open-mx/lib/libopen-mx.so.0(omx_connect+0x8c) >> [0x7fb58741a27c] >> [limulus:04448] [ 9] /usr/open-mx/lib/libopen-mx.so.0(mx_connect+0x15) >> [0x7fb587425865] >> [limulus:04448] [10] >> /opt/mpi/openmpi-gnu4/lib64/libmpi.so.1(mca_btl_mx_proc_connect+0x5e) >> [0x7fb5876fe40e] >> [limulus:04448] [11] >> /opt/mpi/openmpi-gnu4/lib64/libmpi.so.1(mca_btl_mx_send+0x2d4) >> [0x7fb5876fbd94] >> [limulus:04448] [12] >> /opt/mpi/openmpi-gnu4/lib64/libmpi.so.1(mca_pml_ob1_send_request_start_prepare+0xcb) >> [0x7fb58777d6fb] >> [limulus:04448] [13] >> /opt/mpi/openmpi-gnu4/lib64/libmpi.so.1(mca_pml_ob1_isend+0x4cb) >> [0x7fb58777509b] >> [limulus:04448] [14] >> /opt/mpi/openmpi-gnu4/lib64/libmpi.so.1(ompi_coll_tuned_bcast_intra_generic+0x37b) >> [0x7fb58770b55b] >> [limulus:04448] [15] >> /opt/mpi/openmpi-gnu4/lib64/libmpi.so.1(ompi_coll_tuned_bcast_intra_binomial+0xd8) >> [0x7fb58770b8b8] >> [limulus:04448] [16] >> /opt/mpi/openmpi-gnu4/lib64/libmpi.so.1(ompi_coll_tuned_bcast_intra_dec_fixed+0xcc) >> [0x7fb587702d8c] >> [limulus:04448] [17] >> /opt/mpi/openmpi-gnu4/lib64/libmpi.so.1(mca_coll_sync_bcast+0x78) >> [0x7fb587712e88] >> [limulus:04448] [18] >> /opt/mpi/openmpi-gnu4/lib64/libmpi.so.1(MPI_Bcast+0x130) [0x7fb5876ce1b0] >> [limulus:04448] [19] cpi(main+0x10b) [0x400cc4] >> [limulus:04448] [20] /lib64/libc.so.6(__libc_start_main+0xfd) >> [0x332461ecdd] >> [limulus:04448] [21] cpi() [0x400ac9] >> [limulus:04448] *** End of error message *** >> Process 2 on limulus >> Process 4 on limulus >> Process 6 on limulus >> Process 1 on n0 >> Process 7 on n0 >> Process 3 on n0 >> Process 5 on n0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 0 with PID 4448 on node limulus exited >> on signal 6 (Aborted). >> -------------------------------------------------------------------------- >> > [root@limulus ~]# ompi_info > Package: Open MPI deadline@limulus Distribution > Open MPI: 1.6.1 > Open MPI SVN revision: r27106 > Open MPI release date: Aug 22, 2012 > Open RTE: 1.6.1 > Open RTE SVN revision: r27106 > Open RTE release date: Aug 22, 2012 > OPAL: 1.6.1 > OPAL SVN revision: r27106 > OPAL release date: Aug 22, 2012 > MPI API: 2.1 > Ident string: 1.6.1 > Prefix: /opt/mpi/openmpi-gnu4-1.6.1 > Configured architecture: x86_64-unknown-linux-gnu > Configure host: limulus > Configured by: deadline > Configured on: Wed Sep 5 15:03:09 EDT 2012 > Configure host: limulus > Built by: deadline > Built on: Wed Sep 5 15:21:19 EDT 2012 > Built host: limulus > C bindings: yes > C++ bindings: yes > Fortran77 bindings: yes (all) > Fortran90 bindings: yes > Fortran90 bindings size: small > C compiler: gcc > C compiler absolute: /usr/bin/gcc > C compiler family name: GNU > C compiler version: 4.4.6 > C++ compiler: g++ > C++ compiler absolute: /usr/bin/g++ > Fortran77 compiler: gfortran > Fortran77 compiler abs: /usr/bin/gfortran > Fortran90 compiler: /usr/bin/gfortran > Fortran90 compiler abs: > C profiling: yes > C++ profiling: yes > Fortran77 profiling: yes > Fortran90 profiling: yes > C++ exceptions: no > Thread support: posix (MPI_THREAD_MULTIPLE: no, progress: no) > Sparse Groups: no > Internal debug support: no > MPI interface warnings: no > MPI parameter check: runtime > Memory profiling support: no > Memory debugging support: no > libltdl support: yes > Heterogeneous support: no > mpirun default --prefix: no > MPI I/O support: yes > MPI_WTIME support: gettimeofday > Symbol vis. support: yes > Host topology support: yes > MPI extensions: affinity example > FT Checkpoint support: no (checkpoint thread: no) > VampirTrace support: yes > MPI_MAX_PROCESSOR_NAME: 256 > MPI_MAX_ERROR_STRING: 256 > MPI_MAX_OBJECT_NAME: 64 > MPI_MAX_INFO_KEY: 36 > MPI_MAX_INFO_VAL: 256 > MPI_MAX_PORT_NAME: 1024 > MPI_MAX_DATAREP_STRING: 128 > MCA backtrace: execinfo (MCA v2.0, API v2.0, Component v1.6.1) > MCA memory: linux (MCA v2.0, API v2.0, Component v1.6.1) > MCA paffinity: hwloc (MCA v2.0, API v2.0, Component v1.6.1) > MCA carto: auto_detect (MCA v2.0, API v2.0, Component v1.6.1) > MCA carto: file (MCA v2.0, API v2.0, Component v1.6.1) > MCA shmem: mmap (MCA v2.0, API v2.0, Component v1.6.1) > MCA shmem: posix (MCA v2.0, API v2.0, Component v1.6.1) > MCA shmem: sysv (MCA v2.0, API v2.0, Component v1.6.1) > MCA maffinity: first_use (MCA v2.0, API v2.0, Component v1.6.1) > MCA maffinity: hwloc (MCA v2.0, API v2.0, Component v1.6.1) > MCA timer: linux (MCA v2.0, API v2.0, Component v1.6.1) > MCA installdirs: env (MCA v2.0, API v2.0, Component v1.6.1) > MCA installdirs: config (MCA v2.0, API v2.0, Component v1.6.1) > MCA sysinfo: linux (MCA v2.0, API v2.0, Component v1.6.1) > MCA hwloc: hwloc132 (MCA v2.0, API v2.0, Component v1.6.1) > MCA dpm: orte (MCA v2.0, API v2.0, Component v1.6.1) > MCA pubsub: orte (MCA v2.0, API v2.0, Component v1.6.1) > MCA allocator: basic (MCA v2.0, API v2.0, Component v1.6.1) > MCA allocator: bucket (MCA v2.0, API v2.0, Component v1.6.1) > MCA coll: basic (MCA v2.0, API v2.0, Component v1.6.1) > MCA coll: hierarch (MCA v2.0, API v2.0, Component v1.6.1) > MCA coll: inter (MCA v2.0, API v2.0, Component v1.6.1) > MCA coll: self (MCA v2.0, API v2.0, Component v1.6.1) > MCA coll: sm (MCA v2.0, API v2.0, Component v1.6.1) > MCA coll: sync (MCA v2.0, API v2.0, Component v1.6.1) > MCA coll: tuned (MCA v2.0, API v2.0, Component v1.6.1) > MCA io: romio (MCA v2.0, API v2.0, Component v1.6.1) > MCA mpool: fake (MCA v2.0, API v2.0, Component v1.6.1) > MCA mpool: rdma (MCA v2.0, API v2.0, Component v1.6.1) > MCA mpool: sm (MCA v2.0, API v2.0, Component v1.6.1) > MCA pml: bfo (MCA v2.0, API v2.0, Component v1.6.1) > MCA pml: cm (MCA v2.0, API v2.0, Component v1.6.1) > MCA pml: csum (MCA v2.0, API v2.0, Component v1.6.1) > MCA pml: ob1 (MCA v2.0, API v2.0, Component v1.6.1) > MCA pml: v (MCA v2.0, API v2.0, Component v1.6.1) > MCA bml: r2 (MCA v2.0, API v2.0, Component v1.6.1) > MCA rcache: vma (MCA v2.0, API v2.0, Component v1.6.1) > MCA btl: self (MCA v2.0, API v2.0, Component v1.6.1) > MCA btl: mx (MCA v2.0, API v2.0, Component v1.6.1) > MCA btl: sm (MCA v2.0, API v2.0, Component v1.6.1) > MCA btl: tcp (MCA v2.0, API v2.0, Component v1.6.1) > MCA mtl: mx (MCA v2.0, API v2.0, Component v1.6.1) > MCA topo: unity (MCA v2.0, API v2.0, Component v1.6.1) > MCA osc: pt2pt (MCA v2.0, API v2.0, Component v1.6.1) > MCA osc: rdma (MCA v2.0, API v2.0, Component v1.6.1) > MCA iof: hnp (MCA v2.0, API v2.0, Component v1.6.1) > MCA iof: orted (MCA v2.0, API v2.0, Component v1.6.1) > MCA iof: tool (MCA v2.0, API v2.0, Component v1.6.1) > MCA oob: tcp (MCA v2.0, API v2.0, Component v1.6.1) > MCA odls: default (MCA v2.0, API v2.0, Component v1.6.1) > MCA ras: cm (MCA v2.0, API v2.0, Component v1.6.1) > MCA ras: gridengine (MCA v2.0, API v2.0, Component v1.6.1) > MCA ras: loadleveler (MCA v2.0, API v2.0, Component v1.6.1) > MCA ras: slurm (MCA v2.0, API v2.0, Component v1.6.1) > MCA rmaps: load_balance (MCA v2.0, API v2.0, Component v1.6.1) > MCA rmaps: rank_file (MCA v2.0, API v2.0, Component v1.6.1) > MCA rmaps: resilient (MCA v2.0, API v2.0, Component v1.6.1) > MCA rmaps: round_robin (MCA v2.0, API v2.0, Component v1.6.1) > MCA rmaps: seq (MCA v2.0, API v2.0, Component v1.6.1) > MCA rmaps: topo (MCA v2.0, API v2.0, Component v1.6.1) > MCA rml: oob (MCA v2.0, API v2.0, Component v1.6.1) > MCA routed: binomial (MCA v2.0, API v2.0, Component v1.6.1) > MCA routed: cm (MCA v2.0, API v2.0, Component v1.6.1) > MCA routed: direct (MCA v2.0, API v2.0, Component v1.6.1) > MCA routed: linear (MCA v2.0, API v2.0, Component v1.6.1) > MCA routed: radix (MCA v2.0, API v2.0, Component v1.6.1) > MCA routed: slave (MCA v2.0, API v2.0, Component v1.6.1) > MCA plm: rsh (MCA v2.0, API v2.0, Component v1.6.1) > MCA plm: slurm (MCA v2.0, API v2.0, Component v1.6.1) > MCA filem: rsh (MCA v2.0, API v2.0, Component v1.6.1) > MCA errmgr: default (MCA v2.0, API v2.0, Component v1.6.1) > MCA ess: env (MCA v2.0, API v2.0, Component v1.6.1) > MCA ess: hnp (MCA v2.0, API v2.0, Component v1.6.1) > MCA ess: singleton (MCA v2.0, API v2.0, Component v1.6.1) > MCA ess: slave (MCA v2.0, API v2.0, Component v1.6.1) > MCA ess: slurm (MCA v2.0, API v2.0, Component v1.6.1) > MCA ess: slurmd (MCA v2.0, API v2.0, Component v1.6.1) > MCA ess: tool (MCA v2.0, API v2.0, Component v1.6.1) > MCA grpcomm: bad (MCA v2.0, API v2.0, Component v1.6.1) > MCA grpcomm: basic (MCA v2.0, API v2.0, Component v1.6.1) > MCA grpcomm: hier (MCA v2.0, API v2.0, Component v1.6.1) > MCA notifier: command (MCA v2.0, API v1.0, Component v1.6.1) > MCA notifier: syslog (MCA v2.0, API v1.0, Component v1.6.1) > > > [root@limulus ~]# /usr/open-mx/bin/omx_info > Open-MX version 1.5.2 > build: deadline@limulus:/raid1/home/deadline/rpms-sl6/BUILD/open-mx-1.5.2 > Mon Sep 10 08:44:16 EDT 2012 > > Found 1 boards (32 max) supporting 32 endpoints each: > limulus:0 (board #0 name eth0 addr e0:69:95:35:d7:71) > managed by driver 'e1000e' > > Peer table is ready, mapper is 00:00:00:00:00:00 > ================================================ > 0) e0:69:95:35:d7:71 limulus:0 > 1) 00:1c:c0:9b:66:d0 n1:0 > 2) 00:1a:4d:4a:bf:85 n0:0 > 3) 00:1a:4d:4a:bf:83 n2:0 > > > > > > > -- > Doug >
