Re: [OMPI users] knem in Open MPI 1.8.3

> "Nathan" == Nathan Hjelm writes: Hi Nathan Nathan> I want to close the loop on this issue. 1.8.5 will address Nathan> it in several ways: Nathan> - knem support in btl/sm has been fixed. A sanity check was Nathan>disabling knem during component

Re: [OMPI users] knem in Open MPI 1.8.3

Le 31/10/2014 00:24, Gus Correa a écrit : > 2) Any recommendation for the values of the > various vader btl parameters? > [There are 12 of them in OMPI 1.8.3! > That is real challenge to get right.] > > Which values did you use in your benchmarks? > Defaults? > Other? > > In particular, is there

Re: [OMPI users] knem in Open MPI 1.8.3

On 10/30/2014 07:32 PM, Ralph Castain wrote: Just for FYI: I believe Nathan misspoke. The new capability is in 1.8.4, which I hope to release next Friday (Nov 7th) Hi Ralph That is even better! Look forward to OMPI 1.8.4. I still would love to hear from Nathan / OMPI team about my remaining

Re: [OMPI users] knem in Open MPI 1.8.3

Just for FYI: I believe Nathan misspoke. The new capability is in 1.8.4, which I hope to release next Friday (Nov 7th) > On Oct 30, 2014, at 4:24 PM, Gus Correa wrote: > > Hi Nathan > > Thank you very much for addressing this problem. > > I read your notes on Jeff's

Re: [OMPI users] knem in Open MPI 1.8.3

Hi Nathan Thank you very much for addressing this problem. I read your notes on Jeff's blog about vader, and that clarified many things that were obscure to me when I first started this thread whining that knem was not working in OMPI 1.8.3. Thank you also for writing that blog post, and for

Re: [OMPI users] knem in Open MPI 1.8.3

I want to close the loop on this issue. 1.8.5 will address it in several ways: - knem support in btl/sm has been fixed. A sanity check was disabling knem during component registration. I wrote the sanity check before the 1.7 release and didn't intend this side-effect. - vader now

Re: [OMPI users] knem in Open MPI 1.8.3

> On Oct 17, 2014, at 12:06 PM, Gus Correa wrote: > > Hi Jeff > > Many thanks for looking into this and filing a bug report at 11:16PM! > > Thanks to Aurelien, Ralph and Nathan for their help and clarifications > also. > > ** > > Related suggestion: > > Add a note

Re: [OMPI users] knem in Open MPI 1.8.3

Hi Jeff Many thanks for looking into this and filing a bug report at 11:16PM! Thanks to Aurelien, Ralph and Nathan for their help and clarifications also. ** Related suggestion: Add a note to the FAQ explaining that in OMPI 1.8 the new (default) btl is vader (and what it is). It was a real

Re: [OMPI users] knem in Open MPI 1.8.3

On Oct 16, 2014, at 1:35 PM, Gus Correa wrote: > and on the MCA parameter file: > > btl_sm_use_knem = 1 I think the logic enforcing this MCA param got broken when we revamped the MCA param system. :-( > I am scratching my head to understand why a parameter with such

Re: [OMPI users] knem in Open MPI 1.8.3

On 10/16/2014 07:32 PM, Jeff Squyres (jsquyres) wrote: Gus -- Can you send the output of configure and your config.log? Hi Jeff. Sure. This is for the OMPI 1.8.3 build with Intel compilers that I've been using to compile and run IMB. The config.log is attached. The configure command and

Re: [OMPI users] knem in Open MPI 1.8.3

Hi Ralph I have clusters with CentOS 6.4, 6.5, and 5.5. OK, completing my table (ran on CentOS 6.4): #bytes #repetitions t[usec] Mbytes/sec 262144 16048.04 5203.93 :OMPI 1.6.5+knem 262144 16063.72 3923.30 :OMPI 1.8.3+vader 262144 160

Re: [OMPI users] knem in Open MPI 1.8.3

Gus -- Can you send the output of configure and your config.log? On Oct 16, 2014, at 4:24 PM, Gus Correa wrote: > On 10/16/2014 05:38 PM, Nathan Hjelm wrote: >> On Thu, Oct 16, 2014 at 05:27:54PM -0400, Gus Correa wrote: >>> Thank you, Aurelien! >>> >>> Aha, "vader

Re: [OMPI users] knem in Open MPI 1.8.3

On 10/16/2014 05:38 PM, Nathan Hjelm wrote: On Thu, Oct 16, 2014 at 05:27:54PM -0400, Gus Correa wrote: Thank you, Aurelien! Aha, "vader btl", that is new to me! I tought Vader was that man dressed in black in Star Wars, Obi-Wan Kenobi's nemesis. That was a while ago, my kids were children,

Re: [OMPI users] knem in Open MPI 1.8.3

You probably have this somewhere below, but what OS are you running? I have CentOS6, and vader works fine for me and is much faster than the sm btl. I can certainly ask to see if someone has time to fix the knem support - if they do, we would definitely include the fix in the 1.8 series. On

Re: [OMPI users] knem in Open MPI 1.8.3

Hi All Back to the original issue of knem in Open MPI 1.8.3. It really seems to be broken. I launched the Intel MPI benchmarks (IMB) job both with '-mca btl ^vader,tcp', and with '-mca btl sm,self,openib'. Both syntaxes seem to have turned off vader (along with tcp), as shown in stderr by

Re: [OMPI users] knem in Open MPI 1.8.3

On 10/16/2014 05:28 PM, Nathan Hjelm wrote: And it doesn't support knem at this time. Probably never will because of the existence of CMA. -Nathan Thanks, Nathan But for the benefit of mere mortals like me who don't share the dark or the bright side of the force, and just need to keep

Re: [OMPI users] knem in Open MPI 1.8.3

On 10/16/2014 04:49 PM, Ralph Castain wrote: > FWIW: vader is the default in 1.8 Yes, Ralph, thank you, I just noticed it in my job's stderr, after Aurelien pointed out that new "vader" thing existed. What a quick promotion: from inexistent to default btl! But what is "vader" after all? Any

Re: [OMPI users] knem in Open MPI 1.8.3

On Thu, Oct 16, 2014 at 05:27:54PM -0400, Gus Correa wrote: > Thank you, Aurelien! > > Aha, "vader btl", that is new to me! > I tought Vader was that man dressed in black in Star Wars, > Obi-Wan Kenobi's nemesis. > That was a while ago, my kids were children, > and Alec Guiness younger than

Re: [OMPI users] knem in Open MPI 1.8.3

And it doesn't support knem at this time. Probably never will because of the existence of CMA. -Nathan On Thu, Oct 16, 2014 at 01:49:09PM -0700, Ralph Castain wrote: > FWIW: vader is the default in 1.8 > > On Oct 16, 2014, at 1:40 PM, Aurélien Bouteiller wrote: > > > Are

Re: [OMPI users] knem in Open MPI 1.8.3

Thank you, Aurelien! Aha, "vader btl", that is new to me! I tought Vader was that man dressed in black in Star Wars, Obi-Wan Kenobi's nemesis. That was a while ago, my kids were children, and Alec Guiness younger than Harrison Ford is today. Oh, how nostalgic code developers can get when it

Re: [OMPI users] knem in Open MPI 1.8.3

FWIW: vader is the default in 1.8 On Oct 16, 2014, at 1:40 PM, Aurélien Bouteiller wrote: > Are you sure you are not using the vader BTL ? > > Setting mca_btl_base_verbose and/or sm_verbose should spit out some knem > initialization info. > > The CMA linux system

Re: [OMPI users] knem in Open MPI 1.8.3

Are you sure you are not using the vader BTL ? Setting mca_btl_base_verbose and/or sm_verbose should spit out some knem initialization info. The CMA linux system (that ships with most 3.1x linux kernels) has similar features, and is also supported in sm. Aurelien -- ~~~ Aurélien

[OMPI users] knem in Open MPI 1.8.3

Dear Open MPI developers Well, I just can't keep my promises for too long ... So, here I am pestering you again, although this time it is not a request for more documentation. Hopefully it is something more legit. I am having trouble using knem with Open MPI 1.8.3, and need your help. I