I think you'd like to know more than just how many procs are local.
E.g., if the chunk or eager limits are changed much, that would impact
how much memory you'd like to allocate.
A phone chat is all right for me, though so far all I've heard is that
no one understands the code!
But, maybe we can nip this one in the bud. How about the following
proposal.
First, what's happening is:
*) the sm BTL (which knows how big the file should be)
calls
*) mca_mpool_base_module_create()
calls
*) mca_mpool_sm_init() (which creates the file)
There is no explicit calling argument to transmit an mpool size through
these function calls, but there is a "resources" argument. This
resources argument appears to be opaque to the intervening function, but
it seems to be understood by both the sm BTL caller and the sm mpool
component callee. Other components appear to have other definitions of
the resources data structure.
So, I propose:
*) In mca/mpool/sm/mpool_sm.h, there is a definition of
mca_mpool_base_resources_t. It has a single field (int32_t mem_node).
How about I add another field here: size_t size.
*) In the sm BTL in sm_btl_first_time_init(), I can set the size of the
mmap file in my "resources" data structure.
*) In mca_mpool_sm_init(), when I determine the mmap file size, I just
look up the resources->size value and use that.
Yes? Clean and proper solution? Does not break other BTLs?
Ralph Castain wrote:
I also know little about that part of the code, but agree that does
seem weird. Seeing as we know how many local procs there are before
we get to this point, I would think we could be smart about our
memory pool size. You might not need to dive into the sm BTL to get
the info you need - if all you need is how many procs are local, that
can be obtained fairly easily.
Be happy to contribute to the chat, if it would be helpful.
On Jan 14, 2009, at 7:43 AM, Jeff Squyres wrote:
Would it be useful to get on the phone and discuss this stuff?
On Jan 14, 2009, at 1:11 AM, Eugene Loh wrote:
Thanks for the reply. I kind of understand, but it's rather
weird. The BTL calls mca_mpool_base_module_create() to create a
pool of memory, but the BTL has no say how big of a pool to create?
E.g., I see that there is a "resources" argument
(mca_mpool_base_resources_t). Maybe that structure should be
expanded to include a "size" field?
On Jan 13, 2009, at 19:22 , Eugene Loh wrote:
With the sm BTL, there is a file that each process mmaps in for
shared memory.
I'm trying to get mpool_sm to size the file appropriately.
mpool_sm creates and mmaps the file, but the size depends on
parameters like eager limit and max frag size that are known by
the btl_sm.
