On Oct 3, 2015, at 10:12 PM, Dahai Guo <dahaiguo2...@yahoo.com> wrote:
>
> Thanks, Jeff. I am trying to understand in detail how Open MPI works in the
> run time. What main functions does it call to select and initialize the coll
> components?
In general, each framework (e.g., coll, pml, ...etc.) is polled during
MPI_INIT. The framework base will open its components and (generally) query
each one. Some frameworks are many-of-many (i.e., every component that wants
to be used will be kept open and considered), other frameworks are one-of-many
(i.e., somehow one component will be selected as "best" and that will be the
only one that will be kept open / potentially used).
The coll framework selection functionality, for example, is in
ompi/mca/coll/base/.
The coll framework is initialized during MPI_INIT via an MCA base generic
framework initialization function. coll is a many-of-many framework; all
components that say that they might be able to be run are kept.
After that, whenever a communicator is created/constructed, the code in
ompi/mca/coll/base/mca_base_comm_select.c is fired. I.e., we select the union
of coll modules that will be used for each communicator. You can mix and match
modules on a single communicator. For example, you can have a bcast from one
module and an alltoall from another module.
Each component is polled for the new communicator and asked if it wants to run.
IIRC, it returns a module if it wants to run on that communicator, or NULL if
it doesn't. I forget offhand whether we stack-rank *all* available modules (in
a sigaction kind of way), or just take the highest-prioritized module for each
collective operation (check the code). Regardless, the function pointers for
each collective operation are set on the communicator itself.
Then when MPI_BCAST is invoked, for example, we just invoke function pointer
that's hanging off the communicator for the bcast operation.
Make sense?
You'll need to look into the code for the details, but this is a 50,000
description that should get you started.
> Using the "helloworld" as an example, how does it select and initialize the
> MPI_Barrier algorithm? which C functions are involved and used in the
> process?
>
> Dahai
>
>
>
> On Friday, October 2, 2015 7:50 PM, Jeff Squyres (jsquyres)
> <jsquy...@cisco.com> wrote:
>
>
> On Oct 2, 2015, at 2:21 PM, Dahai Guo <dahaiguo2...@yahoo.com> wrote:
> >
> > Is there any way to trace open mpi internal function calls in a MPI user
> > program?
>
> Unfortunately, not easily -- other than using a debugger, for example.
>
> > If so, can any one explain it with an example? such as helloworld? I build
> > open MPI with the VampirTrace options, and compile the following program
> > with picc-vt,. but I didn't get any tracing info.
>
> Open MPI is a giant state machine -- MPI_INIT, for example, invokes slightly
> fewer than a bazillion functions (e.g., it initializes every framework and
> many components/plugins).
>
> Is there something in particular that you're looking for / want to know about?
>
> > Thanks
> >
> > D. G.
> >
> > #include <stdio.h>
> > #include <mpi.h>
> >
> >
> > int main (int argc, char **argv)
> > {
> > int rank, size;
> >
> > MPI_Init (&argc, &argv);
> > MPI_Comm_rank (MPI_COMM_WORLD, &rank);
> > MPI_Comm_size (MPI_COMM_WORLD, &size);
> > printf( "Hello world from process %d of %d\n", rank, size );
> > MPI_Barrier(MPI_COMM_WORLD);
> > MPI_Finalize();
> > return 0;
> > }
> >
> > _______________________________________________
> > devel mailing list
> > de...@open-mpi.org
> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> > Link to this post:
> > http://www.open-mpi.org/community/lists/devel/2015/10/18125.php
>
>
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
>
--
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/
