Hello Galen,
Finally I've got some time to look through the new code.
I have couple of notes. In pml_ob1_rdma.c you try to merge
registrations in the number of places. The code looks like this:
btl_mpool->mpool_deregister(btl_mpool, reg);
btl_mpool->mpool_register(btl_mpool,
new_base,
new_len,
MCA_MPOOL_FLAGS_CACHE,
®);
How do you know reg is not in use? You can't deregister it if somebody
is using the registration!
Also I thought about merging registration and I am not sure this is such
a good idea. The registration may grow to large and you will not be able
to shrink it if only small part of it is in use. This may cause the waste
of memory.
In mca_mpool_base_registration_t structure you save base/bound in byte
granularity, but we know that kernel works in much coarse resolution.
Why not to exploit this fact. We can round base/bound to page boundaries.
We are going to pin this memory anyway. In my patch I introduced
mpool_pageshift for this.
--
Gleb.
