On 5/18/07 2:06 PM, "Andrew Friedley" <afrie...@open-mpi.org> wrote:
> Why not use the binomial mode? I assume it is faster?
Yes, but it doesn't work right this minute (should be fixed soon), and we
would prefer to take a small step first. Linear doesn't require any major
code change, while binomial requires we more significant changes,
particularly regarding how the orteds handle the buffers for
re-transmission. Hence, binomial represents a higher degree of risk.
We have tested linear on a significant range of environments and we
therefore expect to see it work cleanly. Binomial has not been tested very
much at this time. Once we have more fully exercised linear, and have had
time to fix and test binomial on more environments, then we can try
switching the default to binomial.
>
> What are the MCA params to control this?
oob_xcast_mode = {"direct", "linear"} ("binomial" will currently generate an
error)
>
> Can you discuss a little bit about the difference in performance we
> might see between the different modes, and why we might use one over the
> other?
Direct is the slowest due to the number of messages involved. Binomial is
the fastest as it more rapidly propagates the startup message across all the
procs.
Linear is a compromise - it gives you better performance than direct when
ppn > 1 (obviously, when ppn=1, there is no benefit at all), but not as good
as binomial because we send the messages to each orted independently
(instead of via a binomial tree method).
>
> Andrew
>
> Ralph H Castain wrote:
>> For the last several months, we have supported three modes of sending the
>> xcast messages used to release MPI processes from their various stage gates:
>>
>> 1. Direct - message sent directly to each process in a serial fashion
>>
>> 2. Linear - message sent serially to the daemon on each node, which then
>> "fans" it out to the application procs on that node
>>
>> 3. Binomial - message sent via binomial tree algorithm to the daemon on each
>> node, which then "fans" it out to the local application procs
>>
>> To maintain historical consistency, we have defaulted to "direct". However,
>> this is not the more scalable mode.
>>
>> We propose to leave all three of these modes in the system, but to change
>> the default on the OMPI trunk to "linear" so that it will be tested more
>> thoroughly by the automated test suite.
>>
>> Please voice any comments and/or objections. Assuming there is agreement, we
>> will make the switch (solely on the OMPI trunk - this will not impact the
>> 1.2 series) on June 1.
>>
>> Thanks
>> Ralph
>>
>>
>> _______________________________________________
>> devel-core mailing list
>> devel-c...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel-core
> _______________________________________________
> devel-core mailing list
> devel-c...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel-core
