Thank you Gilles. Saliya
On Thu, Jul 9, 2015 at 1:32 AM, Gilles Gouaillardet <gil...@rist.or.jp> wrote: > Saliya, > > there are several things here : > 1) which collective module is used ? > 2) if the tuned collective module is used, then which algo is used ? > 3) which btl is used ? > > First, btl is independent of the collective module. > That means that if you do a collective operation, intra node > communications will (likely) use sm or vader btl which is optimized for > shared memory, and openib/tcp/whatever for inter nodes communications. > > There is a collective module called coll_sm, and if i understand > correctly, it works only on single node communicators, and avoid using any > btl if possible. > > collective modules have different priorities and they do not necessarily > implement all collective operations. > for example, the inter module do not implement barriers on an intra > communicator. conversely, the tuned module do not implement barrier on an > inter communicator. > > in most cases (e.g. default config + intra communicator) the tuned > collective module is used. > each operation has several implementation and they are chosen based on > communicator size and message size. this can be overriden by environment > variable and config file as previously described by George. > > Last but not least, some collective modules (hierarch, ml, ?) implement > hierarchical collective, which means they should be optimized for multi > node / multi tasks per node. > that being said, ml is not production ready, and i am not sure wheter > hierarch is actively maintained) > > i hope this helps > > Gilles > > > On 7/9/2015 5:37 AM, Saliya Ekanayake wrote: > > Hi, > > I see the same collective operation (say allgatherv) implemented in > different ways under tuned, sm, and inter packages. I read from the > documentation [1] that these get picked up depending on the transport. > > Say I run 12 procs per node on 2 nodes totaling 24 procs. If I call > allGatherv collective, will it pick shared memory version to communicate > between procs in the same node and use another for inter node > communication? If so, how can I know/control this? > > Also, if I force the algorithm as, > > coll_tuned_use_dynamic_rules = 1 > coll_tuned_allgatherv_algorithm = 3 > > will it not get the advantage of shared memory? > > [1] https://www.open-mpi.org/faq/?category=sm > > Thank you, > Saliya > > -- > Saliya Ekanayake > Ph.D. Candidate | Research Assistant > School of Informatics and Computing | Digital Science Center > Indiana University, Bloomington > Cell 812-391-4914 > http://saliya.org > > > _______________________________________________ > users mailing listus...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/07/27265.php > > > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/07/27267.php > -- Saliya Ekanayake Ph.D. Candidate | Research Assistant School of Informatics and Computing | Digital Science Center Indiana University, Bloomington Cell 812-391-4914 http://saliya.org
