[OMPI devel] [RFC] Sparse group implementation

2007-07-25 Thread Jeff Squyres
WHAT:Merge the sparse groups work to the trunk; get the community's opinion on one remaining issue. WHY: For large MPI jobs, it can be memory-prohibitive to fully represent dense groups; you can save a lot of space by having "sparse" representations of groups tha

Re: [OMPI devel] [RFC] Sparse group implementation

2007-07-25 Thread Richard Graham
This is good work, so I am happy to see it come over. My initial understanding was that there would be compile time protection for this. In the absence of this, I think we need to see performance data on a variety of communication substrates. It seems like a latency measurement is, perhaps, t

Re: [OMPI devel] [RFC] Sparse group implementation

2007-07-25 Thread Brian Barrett
I have an even bigger objection than Rich. It's near impossible to measure the latency impact of something like this, but it does have an additive effect. It doesn't make sense to have all that code in the critical path for systems where it's not needed. We should leave the compile time

Re: [OMPI devel] MPI_ALLOC_MEM warning when requesting 0 (zero) bytes

2007-07-25 Thread Lisandro Dalcin
On 7/23/07, Jeff Squyres wrote: I think that this will require a little tomfoolery to fix properly because we can't simply return NULL (you can't expect to use the pointer that we return to store anything, but you should be able to expect to be able to dereference it without seg faulting). Exc

Re: [OMPI devel] [OMPI svn] svn:open-mpi r15606

2007-07-25 Thread George Bosilca
Do we really need this one ? It look more like dead code forever to me than anything else. On one hand we're claiming that we don't use any blocking pooling inside (and therefore Open MPI use 100% of the CPU) because a cluster is dedicated to performance gathering, and on the other we allow

Re: [OMPI devel] [OMPI svn] svn:open-mpi r15606

2007-07-25 Thread Jeff Squyres
Aside from updating the TCP BTL to use the not-deprecated MCA parameter interface functions, the new functionality introduced in this patch is about 5 lines of code (1 new member variable in the component struct, one new call to the MCA param register function, then use the new component.me

Re: [OMPI devel] MPI_ALLOC_MEM warning when requesting 0 (zero) bytes

2007-07-25 Thread Jeff Squyres
Be sure to read this thread in order -- the conclusion of the thread was that we now actually *do* return NULL, per POSIX advice. On Jul 25, 2007, at 10:52 AM, Lisandro Dalcin wrote: On 7/23/07, Jeff Squyres wrote: I think that this will require a little tomfoolery to fix properly because

Re: [OMPI devel] Hostfiles - yet again

2007-07-25 Thread Aurelien Bouteiller
Hi Ralph and everyone, I just want to make sure the proposed usecases does not break one of the current open MPI feature I require. For FT purposes, I need to get some specific hosts (lets say with a better MTBF). Those hosts are not part of the MPI_COMM_WORLD but are used to deploy FT service

Re: [OMPI devel] [RFC] Sparse group implementation

2007-07-25 Thread Jeff Squyres
On Jul 25, 2007, at 10:39 AM, Brian Barrett wrote: I have an even bigger objection than Rich. It's near impossible to measure the latency impact of something like this, but it does have an additive effect. It doesn't make sense to have all that code in the critical path for systems where it's

Re: [OMPI devel] [RFC] Sparse group implementation

2007-07-25 Thread Brian Barrett
On Jul 25, 2007, at 2:56 PM, Jeff Squyres wrote: On Jul 25, 2007, at 10:39 AM, Brian Barrett wrote: I have an even bigger objection than Rich. It's near impossible to measure the latency impact of something like this, but it does have an additive effect. It doesn't make sense to have all tha

Re: [OMPI devel] [RFC] Sparse group implementation

2007-07-25 Thread Jeff Squyres
On Jul 25, 2007, at 5:07 PM, Brian Barrett wrote: It just adds a lot of #if's throughout the code. Other than that, there's no reason to remove it. I agree, lots of #ifs are bad. But I guess I don't see the problem. The only real important thing people were directly accessing in the ompi_gro

Re: [OMPI devel] [RFC] Sparse group implementation

2007-07-25 Thread Brian Barrett
On Jul 25, 2007, at 3:14 PM, Jeff Squyres wrote: On Jul 25, 2007, at 5:07 PM, Brian Barrett wrote: It just adds a lot of #if's throughout the code. Other than that, there's no reason to remove it. I agree, lots of #ifs are bad. But I guess I don't see the problem. The only real important t

[OMPI devel] openib btl / --enable-progress-threads

2007-07-25 Thread Jeff Squyres
I notice that --enable-progress-threads with the openib btl is currently broken (i.e., it won't even compile). I know that it doesn't run (because of other threading issues currently being worked on), but a broken compilation does prevent other kinds of testing. The problem is that the ope

Re: [OMPI devel] [RFC] Sparse group implementation

2007-07-25 Thread Mohamad Chaarawi
In the current code, almost all #ifs are due to the fact that we don't want to add the extra memory by the sparse parameters that are added to the group structure. The additional parameters are 5 pointers and 3 integers. If nobody objects, i would actually keep those extra parameters, even if spars

Re: [OMPI devel] [RFC] Sparse group implementation

2007-07-25 Thread Mohamad Chaarawi
Sorry the additional parameters are 5 integers and 3 pointers.. Mohamad On Wed, July 25, 2007 9:45 pm, Mohamad Chaarawi wrote: > In the current code, almost all #ifs are due to the fact that we don't > want to add the extra memory by the sparse parameters that are added to > the group structure.