Hi, I have a communicator, say *comm*, and some shuffling of data takes place within its nodes.
I have implemented the shuffling with broadcasts but now I am trying to experiment with MPI_Allgather() and MPI_Allgatherv(). For demonstration purposes I am adding here a small part of the C++ code. You can assume that *no_keys* keys each of size *keysize *unsigned char are locally stored at some structure named *endata, *at each node. *unsigned keysize = 100;* *unsigned long long no_keys = 10;* *// unsigned long long bytes_send_count = no_keys*keysize; * *int bytes_send_count = (int)no_keys*keysize; * *unsigned int commSize = (unsigned)comm.Get_size();* *// unsigned long long* recv_counts = new unsigned long long[commSize];* *int* recv_counts = new int[commSize];* *// unsigned long long* displs = new unsigned long long[commSize];* *int* displs = new int[commSize]; * *//Εxchange amount of data* *// comm.Allgather(&bytes_send_count, 1, MPI::UNSIGNED_LONG_LONG, recv_counts, 1, MPI::UNSIGNED_LONG_LONG); * *comm.Allgather(&bytes_send_count, 1, MPI::INT, recv_counts, 1, MPI::INT); * *unsigned long long total = 0; * *for(unsigned int i = 0; i < commSize; i++){ * * displs[i] = total; * * total += recv_counts[i]; * *}* *unsigned char* recv_buf = new unsigned char[total];* *//Exchange actual data* *comm.Allgatherv(&endata, no_keys*keysize, MPI::UNSIGNED_CHAR, recv_buf, recv_counts, displs, MPI::UNSIGNED_CHAR);* My problem is that the number of keys is actually big and cannot fit into an *int*, that's why it is defined as an *unsigned long long*. So, what I would like to do is basically the commented lines of code but I get the error that there is no match to this function if I use *unsigned long long*. Is this true, and if yes why is that? I don't understand why Bcast() supports unsigned long long while Allgatherv() does not. I would like to avoid setting up multiple Allgatherv() calls for this since the number of connections that are initiated based on my algorithm is quite big and I am afraid that this will create further delays. Actually, this is the reason I am trying to replace Bcast() and try other things. I am using Open MPI 2.1.2 and testing on a single computer with 7 MPI processes. The ompi_info is the attached file.
Package: Open MPI ubuntu@ip-172-31-30-250 Distribution Open MPI: 2.1.2 Open MPI repo revision: v2.1.1-188-g6157ed8 Open MPI release date: Sep 20, 2017 Open RTE: 2.1.2 Open RTE repo revision: v2.1.1-188-g6157ed8 Open RTE release date: Sep 20, 2017 OPAL: 2.1.2 OPAL repo revision: v2.1.1-188-g6157ed8 OPAL release date: Sep 20, 2017 MPI API: 3.1.0 Ident string: 2.1.2 Prefix: /usr/local Configured architecture: x86_64-unknown-linux-gnu Configure host: ip-172-31-30-250 Configured by: ubuntu Configured on: Sun Nov 19 22:08:57 UTC 2017 Configure host: ip-172-31-30-250 Built by: root Built on: Sun Nov 19 22:18:49 UTC 2017 Built host: ip-172-31-30-250 C bindings: yes C++ bindings: yes Fort mpif.h: yes (all) Fort use mpi: yes (full: ignore TKR) Fort use mpi size: deprecated-ompi-info-value Fort use mpi_f08: yes Fort mpi_f08 compliance: The mpi_f08 module is available, but due to limitations in the gfortran compiler, does not support the following: array subsections, direct passthru (where possible) to underlying Open MPI's C functionality Fort mpi_f08 subarrays: no Java bindings: no Wrapper compiler rpath: runpath C compiler: gcc C compiler absolute: /usr/bin/gcc C compiler family name: GNU C compiler version: 5.4.0 C++ compiler: g++ C++ compiler absolute: /usr/bin/g++ Fort compiler: gfortran Fort compiler abs: /usr/bin/gfortran Fort ignore TKR: yes (!GCC$ ATTRIBUTES NO_ARG_CHECK ::) Fort 08 assumed shape: yes Fort optional args: yes Fort INTERFACE: yes Fort ISO_FORTRAN_ENV: yes Fort STORAGE_SIZE: yes Fort BIND(C) (all): yes Fort ISO_C_BINDING: yes Fort SUBROUTINE BIND(C): yes Fort TYPE,BIND(C): yes Fort T,BIND(C,name="a"): yes Fort PRIVATE: yes Fort PROTECTED: yes Fort ABSTRACT: yes Fort ASYNCHRONOUS: yes Fort PROCEDURE: yes Fort USE...ONLY: yes Fort C_FUNLOC: yes Fort f08 using wrappers: yes Fort MPI_SIZEOF: yes C profiling: yes C++ profiling: yes Fort mpif.h profiling: yes Fort use mpi profiling: yes Fort use mpi_f08 prof: yes C++ exceptions: no Thread support: posix (MPI_THREAD_MULTIPLE: no, OPAL support: yes, OMPI progress: no, ORTE progress: yes, Event lib: yes) Sparse Groups: no Internal debug support: yes MPI interface warnings: yes MPI parameter check: runtime Memory profiling support: no Memory debugging support: no dl support: yes Heterogeneous support: no mpirun default --prefix: no MPI I/O support: yes MPI_WTIME support: native Symbol vis. support: yes Host topology support: yes MPI extensions: affinity, cuda MPI_MAX_PROCESSOR_NAME: 256 MPI_MAX_ERROR_STRING: 256 MPI_MAX_OBJECT_NAME: 64 MPI_MAX_INFO_KEY: 36 MPI_MAX_INFO_VAL: 256 MPI_MAX_PORT_NAME: 1024 MPI_MAX_DATAREP_STRING: 128 MCA allocator: basic (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA allocator: bucket (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA backtrace: execinfo (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA btl: vader (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA btl: sm (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA btl: openib (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA btl: tcp (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA btl: self (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA dl: dlopen (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA event: libevent2022 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA hwloc: hwloc1112 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA if: posix_ipv4 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA if: linux_ipv6 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA installdirs: env (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA installdirs: config (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA memchecker: valgrind (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA memory: patcher (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA mpool: hugepage (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA patcher: overwrite (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA pmix: pmix112 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA pstat: linux (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rcache: grdma (MCA v2.1.0, API v3.3.0, Component v2.1.2) MCA sec: basic (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA shmem: posix (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA shmem: mmap (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA shmem: sysv (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA timer: linux (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA dfs: test (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA dfs: app (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA dfs: orted (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA errmgr: default_app (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA errmgr: default_hnp (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA errmgr: default_orted (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA errmgr: default_tool (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA ess: hnp (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA ess: pmi (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA ess: tool (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA ess: env (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA ess: singleton (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA ess: slurm (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA filem: raw (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA grpcomm: direct (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA iof: hnp (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA iof: mr_orted (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA iof: orted (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA iof: mr_hnp (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA iof: tool (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA notifier: syslog (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA odls: default (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA oob: tcp (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA oob: usock (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA oob: ud (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA plm: slurm (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA plm: rsh (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA plm: isolated (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA ras: simulator (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA ras: loadleveler (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA ras: slurm (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: round_robin (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: ppr (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: resilient (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: staged (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: rank_file (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: seq (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rmaps: mindist (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rml: oob (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA routed: radix (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA routed: binomial (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA routed: direct (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA routed: debruijn (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rtc: freq (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA rtc: hwloc (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA schizo: ompi (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: hnp (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: dvm (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: staged_orted (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: orted (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: novm (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: tool (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: staged_hnp (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA state: app (MCA v2.1.0, API v1.0.0, Component v2.1.2) MCA bml: r2 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: inter (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: sm (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: libnbc (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: basic (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: self (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: tuned (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA coll: sync (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fbtl: posix (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fcoll: static (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fcoll: dynamic (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fcoll: two_phase (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fcoll: individual (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fcoll: dynamic_gen2 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA fs: ufs (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA io: romio314 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA io: ompio (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA osc: sm (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA osc: rdma (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA osc: pt2pt (MCA v2.1.0, API v3.0.0, Component v2.1.2) MCA pml: v (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA pml: ob1 (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA pml: cm (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA rte: orte (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA sharedfp: lockedfile (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA sharedfp: individual (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA sharedfp: sm (MCA v2.1.0, API v2.0.0, Component v2.1.2) MCA topo: basic (MCA v2.1.0, API v2.2.0, Component v2.1.2) MCA vprotocol: pessimist (MCA v2.1.0, API v2.0.0, Component v2.1.2)
_______________________________________________ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users