Re: [OMPI devel] RFC: job size info in OPAL

2014-07-31 Thread Ralph Castain
en you want to initialize >>> for big mailboxes (IBer's can think many large buffers posted as RX WQEs). >>> But for very large jobs, with possibly highly connected communication >>> pattern, >>> you want very small mailboxes. >>> >>> How

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-31 Thread Nathan Hjelm
s, then you want to initialize > > for big mailboxes (IBer's can think many large buffers posted as RX WQEs). > > But for very large jobs, with possibly highly connected communication > > pattern, > > you want very small mailboxes. > > > > Howard > > &g

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-31 Thread Ralph Castain
bly highly connected communication pattern, > you want very small mailboxes. > > Howard > > > -Original Message- > From: devel [mailto:devel-boun...@open-mpi.org] On Behalf Of George Bosilca > Sent: Thursday, July 31, 2014 9:09 AM > To: Open MPI Developers >

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-31 Thread Nathan Hjelm
I do not like the fact that add_procs is called with every proc in the MPI_COMM_WORLD. That needs to change, so, I will not rely on the number of procs being added being the same as the world or universe size. -Nathan On Thu, Jul 31, 2014 at 09:22:00AM -0600, George Bosilca wrote: >I definiti

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-31 Thread Pritchard Jr., Howard
elopers Subject: Re: [OMPI devel] RFC: job size info in OPAL What is your definition of "global job size"? George. On Jul 31, 2014, at 11:06 , Pritchard Jr., Howard wrote: > Hi Folks, > > I think given the way we want to use the btl's in lower levels like > opal,

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-31 Thread George Bosilca
I definitively think you misunderstood this scope of this RFC. The information that is so important to you to configure the mailbox size is available to you when you need it. This information is made available by the PML through the call to add_procs, which comes with all the procs in the MPI_CO

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-31 Thread Nathan Hjelm
s add in a hack to probe the apps placement info file and > > scale the smsg blocks by number of nodes rather than number of ranks. > > > > Howard > > > > > > -Original Message- > > From: devel [mailto:devel-boun...@open-mpi.org] On Behalf Of

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-31 Thread Ralph Castain
ever that can be done in ugni for now. >> I can always add in a hack to probe the apps placement info file and >> scale the smsg blocks by number of nodes rather than number of ranks. >> >> Howard >> >> >> -Original Message----- >> From: devel

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-31 Thread George Bosilca
l-boun...@open-mpi.org] On Behalf Of Nathan Hjelm > Sent: Thursday, July 31, 2014 8:58 AM > To: Open MPI Developers > Subject: Re: [OMPI devel] RFC: job size info in OPAL > > > +2^1000 > > This information is absolutely necessary at this point. If someone has a > bet

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-31 Thread Pritchard Jr., Howard
des rather than number of ranks. Howard -Original Message- From: devel [mailto:devel-boun...@open-mpi.org] On Behalf Of Nathan Hjelm Sent: Thursday, July 31, 2014 8:58 AM To: Open MPI Developers Subject: Re: [OMPI devel] RFC: job size info in OPAL +2^1000 This information is absolutely

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-31 Thread Nathan Hjelm
+2^1000 This information is absolutely necessary at this point. If someone has a better solution they can provide it as an alternative RFC. Until then this is how it should be done... Otherwise we loose uGNI support on the trunk. Because we ARE NOT going to remove the mailbox size optimizatio

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-30 Thread Ralph Castain
On Jul 30, 2014, at 5:49 PM, George Bosilca wrote: > > On Jul 30, 2014, at 20:37 , Ralph Castain wrote: > >> >> On Jul 30, 2014, at 5:25 PM, George Bosilca wrote: >> >>> >>> On Jul 30, 2014, at 18:00 , Jeff Squyres (jsquyres) >>> wrote: >>> WHAT: Should we make the job size (i.e.,

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-30 Thread George Bosilca
On Jul 30, 2014, at 20:37 , Ralph Castain wrote: > > On Jul 30, 2014, at 5:25 PM, George Bosilca wrote: > >> >> On Jul 30, 2014, at 18:00 , Jeff Squyres (jsquyres) >> wrote: >> >>> WHAT: Should we make the job size (i.e., initial number of procs) available >>> in OPAL? >>> >>> WHY: At l

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-30 Thread Ralph Castain
On Jul 30, 2014, at 5:25 PM, George Bosilca wrote: > > On Jul 30, 2014, at 18:00 , Jeff Squyres (jsquyres) > wrote: > >> WHAT: Should we make the job size (i.e., initial number of procs) available >> in OPAL? >> >> WHY: At least 2 BTLs are using this info (*more below) >> >> WHERE: usnic

Re: [OMPI devel] RFC: job size info in OPAL

2014-07-30 Thread George Bosilca
On Jul 30, 2014, at 18:00 , Jeff Squyres (jsquyres) wrote: > WHAT: Should we make the job size (i.e., initial number of procs) available > in OPAL? > > WHY: At least 2 BTLs are using this info (*more below) > > WHERE: usnic and ugni > > TIMEOUT: there's already been some inflammatory emails

[OMPI devel] RFC: job size info in OPAL

2014-07-30 Thread Jeff Squyres (jsquyres)
WHAT: Should we make the job size (i.e., initial number of procs) available in OPAL? WHY: At least 2 BTLs are using this info (*more below) WHERE: usnic and ugni TIMEOUT: there's already been some inflammatory emails about this; let's discuss next Tuesday on the teleconf: Tue, 5 Aug 2014 MORE