Hi, consider a setup of one MPI sender process unicasting J messages to each of N MPI receiver processes, i.e., the total number of transmissions is J*N. Each transmission is a block of a matrix, which has been split both horizontally and vertically. Each block has been stored as an element of a 2D list, i.e. a list of J lists, each of which has N elements. I won't go into the details of the Python code (please see attached) but I am pasting the problematic part here:
On the sender's level: *reqA = [None] * J * N* *for i in range(J):* * for j in range(N):* * reqA[i*N+j] = comm.Isend([np.ascontiguousarray(Ahv[i][j//k][j%k]), MPI.INT <http://MPI.INT>], dest=j+1, tag=15)* *MPI.Request.Waitall(reqA)* On the receiver's level: *A = []* *rA = [None] * J* *for i in range(J):* * A.append(np.empty_like(np.matrix([[0]*(n//k) for j in range(m//q)])))* * rA[i] = comm.Irecv(A[i], source=0, tag=15)* *MPI.Request.Waitall(rA)* *#test* *for i in range(J):* * print "For job %d, worker %d received splits of A, b" % (i, comm.Get_rank()-1)* * print(A[i])* There are some other transmissions/requests in between which can be found in the attached file. I do not think they interfere with those, though. When I print all of the received blocks, they look like some memory trash and none of them is the same as the blocks that were sent. I am using Python 2.7.14 and Open MPI 2.1.2 (I have also attached the output of ompi_info). Thanks, Kostas Konstantinidis
Package: Open MPI kostas@kostas-VirtualBox Distribution Open MPI: 2.1.2 Open MPI repo revision: v2.1.1-188-g6157ed8 Open MPI release date: Sep 20, 2017 Open RTE: 2.1.2 Open RTE repo revision: v2.1.1-188-g6157ed8 Open RTE release date: Sep 20, 2017 OPAL: 2.1.2 OPAL repo revision: v2.1.1-188-g6157ed8 OPAL release date: Sep 20, 2017 MPI API: 3.1.0 Ident string: 2.1.2 Prefix: /usr/local Configured architecture: x86_64-unknown-linux-gnu Configure host: kostas-VirtualBox Configured by: kostas Configured on: Sun May 6 22:54:43 CDT 2018 Configure host: kostas-VirtualBox Built by: root Built on: Sun May 6 23:03:58 CDT 2018 Built host: kostas-VirtualBox C bindings: yes C++ bindings: yes Fort mpif.h: yes (all) Fort use mpi: yes (full: ignore TKR) Fort use mpi size: deprecated-ompi-info-value Fort use mpi_f08: yes Fort mpi_f08 compliance: The mpi_f08 module is available, but due to limitations in the gfortran compiler, does not support the following: array subsections, direct passthru (where possible) to underlying Open MPI's C functionality Fort mpi_f08 subarrays: no Java bindings: no Wrapper compiler rpath: runpath C compiler: gcc C compiler absolute: /usr/bin/gcc C compiler family name: GNU C compiler version: 5.4.0 C++ compiler: g++ C++ compiler absolute: /usr/bin/g++ Fort compiler: gfortran Fort compiler abs: /usr/bin/gfortran Fort ignore TKR: yes (!GCC$ ATTRIBUTES NO_ARG_CHECK ::) Fort 08 assumed shape: yes Fort optional args: yes Fort INTERFACE: yes Fort ISO_FORTRAN_ENV: yes Fort STORAGE_SIZE: yes Fort BIND(C) (all): yes Fort ISO_C_BINDING: yes Fort SUBROUTINE BIND(C): yes Fort TYPE,BIND(C): yes Fort T,BIND(C,name="a"): yes Fort PRIVATE: yes Fort PROTECTED: yes Fort ABSTRACT: yes Fort ASYNCHRONOUS: yes Fort PROCEDURE: yes Fort USE...ONLY: yes Fort C_FUNLOC: yes Fort f08 using wrappers: yes Fort MPI_SIZEOF: yes C profiling: yes C++ profiling: yes Fort mpif.h profiling: yes Fort use mpi profiling: yes Fort use mpi_f08 prof: yes C++ exceptions: no Thread support: posix (MPI_THREAD_MULTIPLE: no, OPAL support: yes, OMPI progress: no, ORTE progress: yes, Event lib: yes) Sparse Groups: no Internal debug support: yes MPI interface warnings: yes MPI parameter check: runtime Memory profiling support: no Memory debugging support: no dl support: yes Heterogeneous support: no mpirun default --prefix: no MPI I/O support: yes MPI_WTIME support: native Symbol vis. support: yes Host topology support: yes MPI extensions: affinity, cuda MPI_MAX_PROCESSOR_NAME: 256 MPI_MAX_ERROR_STRING: 256 MPI_MAX_OBJECT_NAME: 64 MPI_MAX_INFO_KEY: 36 MPI_MAX_INFO_VAL: 256 MPI_MAX_PORT_NAME: 1024 MPI_MAX_DATAREP_STRING: 128 MCA allocator: basic (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA allocator: bucket (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA backtrace: execinfo (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA btl: tcp (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA btl: sm (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA btl: openib (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA btl: vader (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA btl: self (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA dl: dlopen (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA event: libevent2022 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA hwloc: hwloc1112 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA if: posix_ipv4 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA if: linux_ipv6 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA installdirs: env (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA installdirs: config (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA memchecker: valgrind (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA memory: patcher (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA mpool: hugepage (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA patcher: overwrite (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA pmix: pmix112 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA pstat: linux (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rcache: grdma (MCA v2.1.0, API v3.3.0, Component v2.1.2) MCA sec: basic (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA shmem: sysv (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA shmem: mmap (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA shmem: posix (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA timer: linux (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA dfs: app (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA dfs: test (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA dfs: orted (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA errmgr: default_app (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA errmgr: default_hnp (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA errmgr: default_orted (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA errmgr: default_tool (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA ess: hnp (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA ess: pmi (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA ess: env (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA ess: tool (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA ess: singleton (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA ess: slurm (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA filem: raw (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA grpcomm: direct (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA iof: mr_hnp (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA iof: tool (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA iof: hnp (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA iof: mr_orted (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA iof: orted (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA notifier: syslog (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA odls: default (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA oob: ud (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA oob: usock (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA oob: tcp (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA plm: slurm (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA plm: isolated (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA plm: rsh (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA ras: simulator (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA ras: slurm (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA ras: loadleveler (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: mindist (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: resilient (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: rank_file (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: round_robin (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: ppr (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: staged (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: seq (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rml: oob (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA routed: direct (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA routed: radix (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA routed: binomial (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA routed: debruijn (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rtc: freq (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA rtc: hwloc (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA schizo: ompi (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: staged_hnp (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: tool (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: novm (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: dvm (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: app (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: hnp (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: orted (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: staged_orted (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA bml: r2 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: basic (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: self (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: sm (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: inter (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: libnbc (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: sync (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: tuned (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fbtl: posix (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fcoll: individual (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fcoll: dynamic (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fcoll: dynamic_gen2 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fcoll: two_phase (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fcoll: static (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fs: ufs (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA io: romio314 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA io: ompio (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA osc: pt2pt (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA osc: sm (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA osc: rdma (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA pml: v (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA pml: cm (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA pml: ob1 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rte: orte (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA sharedfp: sm (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA sharedfp: individual (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA sharedfp: lockedfile (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA topo: basic (MCA v2.1.0, API v2.2.0, Component v2.1.2) MCA vprotocol: pessimist (MCA v2.1.0, API v2.0.0, Component v2.1.2)
uncoded.py
Description: Binary data
_______________________________________________ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users