On Apr 2, 2008, at 4:12 PM, Gleb Natapov wrote:
I can specify
different openib_if_include values for different procs on the same
host.
I know you *can*, but it is certainly uncommon. The common case is
Uncommon - yes, but do you what to make it unsupported?
No, there's no need for that.
that it's the same for all procs on all hosts. I guess there's a few
cases:
1. homogeneous include/exclude, no carto: send all in node info; no
proc info
2. homogeneous include/exclude, carto is used: send all ports in node
info; send index in proc info for which node info port index it
will use
This may actually increase modex size. Think about two procs using two
different hcas. We'll send all the data we send today + indexes.
It'll increase it compared to the optimization that we're about to
make. But it will certainly be a large decrease compared to what
we're doing today (see the spreadsheet that I sent last week).
Indeed, we can even put in the optimization that if there's only one
process on a host, it can only publish the ports that it will use (and
therefore there's no need for the proc data).
3. heterogeneous include/exclude, no cart: need user to tell us that
this situation exists (e.g., use another MCA param), but then is same
as #2
4. heterogeneous include/exclude, cart is used, same as #3
Right?
Looks like it. FWIW I don't like the idea to code all those special
cases. The way it works now I can be pretty sure that any crazy setup
I'll come up with will work.
And so it will with the new scheme. The only place it won't work is
if the user specifies a heterogeneous include/exclude (i.e., we'll
require that the user tells us when they do that), which nobody does.
I guess I don't see the problem...?
By the way how much data are moved during modex stage? What if modex
will use compression?
The spreadsheet I listed was just the openib part of the modex, and it
was fairly hefty. I have no idea how well (or not) it would compress.
--
Jeff Squyres
Cisco Systems
