Re: [OMPI devel] Any plans to support Intel MIC (Xeon Phi) in Open-MPI?

2013-05-02 Thread Ralph Castain
Hmmm...well, a few points here. First, the Phi's sadly don't show up in the hwloc tree as they apparently are hidden behind the PCIe bridge. I don't know if there is a way for hwloc to "probe" and find processors on PCI cards, but that's something I'll have to defer to Jeff and Brice. So the fi

Re: [OMPI devel] Any plans to support Intel MIC (Xeon Phi) in Open-MPI?

2013-05-02 Thread Paul Hargrove
Jeff, I know Intel MPI (MPICH based) "just works" with Phi, but you need to do things like: mpirun –n 2 –host cpu host.exe : –n 4 –host mic0 mic.exe if you want to use the Phi for more than just kernel-offload (in which case they won't have/need an MPI rank). So, launch procs is PART of the pro

Re: [OMPI devel] Any plans to support Intel MIC (Xeon Phi) in Open-MPI?

2013-05-02 Thread Jeff Squyres (jsquyres)
I know the MPICH guys did a bunch of work to support the Phi's. I don't know exactly what that means (I haven't read their docs about this stuff), but I suspect that it's more than just launching MPI processes on them... On May 2, 2013, at 8:54 PM, Paul Hargrove wrote: > Ralph, > > I am not

Re: [OMPI devel] Any plans to support Intel MIC (Xeon Phi) in Open-MPI?

2013-05-02 Thread Christopher Samuel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/05/13 10:47, Ralph Castain wrote: > We had something similar at one time - I developed it for the > Roadrunner cluster so you could run MPI tasks on the GPUs. Worked > well, but eventually fell into disrepair due to lack of use. OK, interesti

Re: [OMPI devel] Any plans to support Intel MIC (Xeon Phi) in Open-MPI?

2013-05-02 Thread Paul Hargrove
Ralph, I am not an expert, by any means, but based on a presentation I heard 4 hours ago: The Xeon and Phi instruction sets have a large intersection, but neither is a subset of the other. In particular, Phi has its own SIMD instructions *instead* of Xeon's MMX, SSEn, etc. There is also on CMPXCH

Re: [OMPI devel] Any plans to support Intel MIC (Xeon Phi) in Open-MPI?

2013-05-02 Thread Ralph Castain
On May 2, 2013, at 5:12 PM, Christopher Samuel wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hi folks, > > The new system we're bringing up has 10 nodes with dual Xeon Phi MIC > cards, are there any plans to support them by launching MPI tasks > directly on the Phis themselves (

[OMPI devel] Any plans to support Intel MIC (Xeon Phi) in Open-MPI?

2013-05-02 Thread Christopher Samuel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi folks, The new system we're bringing up has 10 nodes with dual Xeon Phi MIC cards, are there any plans to support them by launching MPI tasks directly on the Phis themselves (rather than just as offload devices for code on the hosts)? All the best

Re: [OMPI devel] Choosing an Open-MPI release for a new cluster

2013-05-02 Thread Christopher Samuel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Ralph, Jeff, Paul, On 02/05/13 14:14, Ralph Castain wrote: > Depends on what you think you might want, and how tolerant you and > your users are about bugs. > > The 1.6 series is clearly more mature and stable. It has nearly > all the MPI-2 stuf

[OMPI devel] BTL TCP failures

2013-05-02 Thread Jeff Squyres (jsquyres)
On my trunk MTT runs, I'm getting a bunch of timed out tests with this message: [node014][[56709,1],31][btl_tcp_endpoint.c:678:mca_btl_tcp_endpoint_complete_connect] connect() to 10.1.0.7 failed: No route to host (113) This appeared to be due to a problem with a switch in my cluster, but the la

Re: [OMPI devel] Choosing an Open-MPI release for a new cluster

2013-05-02 Thread Paul Hargrove
On Thu, May 2, 2013 at 5:55 AM, Jeff Squyres (jsquyres) wrote: > Given that we know there are compile bugs with XRC in all the v1.7 > releases so far (which seems to show how few people are actually using > XRC...), there are definitely "growing pains", as Ralph mentioned. Jeff may be a little

Re: [OMPI devel] Choosing an Open-MPI release for a new cluster

2013-05-02 Thread Jeff Squyres (jsquyres)
On May 2, 2013, at 12:14 AM, Ralph Castain wrote: > Personally, even though I'm one of the 1.7 release managers, I'm a little > leery of recommending it for a production installation until we get further > down the road. You might consider installing 1.6 as your "base" release, and > then offe

Re: [OMPI devel] Using external libevent

2013-05-02 Thread Jeff Squyres (jsquyres)
On May 1, 2013, at 10:32 PM, Orion Poplawski wrote: > Great! I'll try to take a look at next week. Hold off on this -- Ralph and I looked at this a bit closer, and the work is not quite complete yet (read: it doesn't work). > I noticed another message about using a threaded libevent after all

Re: [OMPI devel] Choosing an Open-MPI release for a new cluster

2013-05-02 Thread Ralph Castain
Depends on what you think you might want, and how tolerant you and your users are about bugs. The 1.6 series is clearly more mature and stable. It has nearly all the MPI-2 stuff now, but no MPI-3. If you think there is something in MPI-3 you might want, then the 1.7 series could be the way to