Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-20 Thread Pavel Shamis
gt; https://centos.pkgs.org/7/epel-x86_64/ucx-1.3.1-1.el7.x86_64.rpm.html > > -Original Message- > From: Peter Kjellström > Sent: Thursday, September 20, 2018 2:11 PM > To: Yossi Itigin > Cc: Open MPI Developers > Subject: Re: [OMPI devel] Announcing Open MPI v4.0.0rc1 > >

Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-20 Thread Yossi Itigin
Subject: Re: [OMPI devel] Announcing Open MPI v4.0.0rc1 On Thu, 20 Sep 2018 09:03:44 + Yossi Itigin wrote: > Hi, > > UCX is on the way into RH distro and will be available and ON by > default (auto-detectable by OMPI build process) automatically in the > near future. That

Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-20 Thread Peter Kjellström
On Thu, 20 Sep 2018 14:18:35 +0200 Peter Kjellström wrote: > On Wed, 19 Sep 2018 16:24:53 + > "Gabriel, Edgar" wrote: ... > > So bottom line, if I do > > > > mpirun –mca btl^openib –mca mtl^ofi …. > > > > my tests finish correctly, although mpirun will still return an > > error. > > I

Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-20 Thread Peter Kjellström
On Wed, 19 Sep 2018 16:24:53 + "Gabriel, Edgar" wrote: > I performed some tests on our Omnipath cluster, and I have a mixed > bag of results with 4.0.0rc1 I've also tried it on our OPA cluster (skylake+centos-7+inbox) with very similar results. > compute-1-1.local.4351PSM2 has not been

Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-20 Thread Peter Kjellström
On Thu, 20 Sep 2018 09:03:44 + Yossi Itigin wrote: > Hi, > > UCX is on the way into RH distro and will be available and ON by > default (auto-detectable by OMPI build process) automatically in the > near future. That is good to hear, already in the upcomming 7.6? > Meanwhile, user can

Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-20 Thread Peter Kjellström
On Tue, 18 Sep 2018 20:49:52 + "Jeff Squyres \(jsquyres\) via devel" wrote: > On Sep 18, 2018, at 3:46 PM, Thananon Patinyasakdikul > wrote: > > > > I tested on our cluster (UTK). I will give a thumb up but I have > > some comments. > > > > What I understand with 4.0. > > - openib btl is

Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-20 Thread Yossi Itigin
yres) via devel Sent: Tuesday, September 18, 2018 11:50 PM To: Open MPI Developers List Cc: Jeff Squyres (jsquyres) Subject: Re: [OMPI devel] Announcing Open MPI v4.0.0rc1 On Sep 18, 2018, at 3:46 PM, Thananon Patinyasakdikul wrote: > > I tested on our cluster (UTK). I will give a thum

Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-19 Thread Cabral, Matias A
tember 19, 2018 10:15 AM To: Open MPI Developers Subject: Re: [OMPI devel] Announcing Open MPI v4.0.0rc1 Mattias, IIRC, OFI BTL only create one EP. If you move it to add_proc, you might need to add some checks to not re-creating EP over and over. Do you think moving EP creation from compo

Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-19 Thread Thananon Patinyasakdikul
the EPs opening to add_procs. > Feedback? > > Number 3 is goes beyond me. > > Thanks, > > _MAC > > <>From: devel [mailto:devel-boun...@lists.open-mpi.org] On Behalf Of > Gabriel, Edgar > Sent: Wednesday, September 19, 2018 9:25 AM > To: Open MPI Develope

Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-19 Thread Cabral, Matias A
From: devel [mailto:devel-boun...@lists.open-mpi.org] On Behalf Of Gabriel, Edgar Sent: Wednesday, September 19, 2018 9:25 AM To: Open MPI Developers Subject: Re: [OMPI devel] Announcing Open MPI v4.0.0rc1 I performed some tests on our Omnipath cluster, and I have a mixed bag of results

Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-19 Thread Gabriel, Edgar
I performed some tests on our Omnipath cluster, and I have a mixed bag of results with 4.0.0rc1 1. Good news, the problems with the psm2 mtl that I reported in June/July seem to be fixed. I still get however a warning every time I run a job with 4.0.0, e.g. compute-1-1.local.4351PSM2

Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-18 Thread Jeff Squyres (jsquyres) via devel
On Sep 18, 2018, at 3:46 PM, Thananon Patinyasakdikul wrote: > > I tested on our cluster (UTK). I will give a thumb up but I have some > comments. > > What I understand with 4.0. > - openib btl is disabled by default (can be turned on by mca) It is disabled by default *for InfiniBand*. It

Re: [OMPI devel] Announcing Open MPI v4.0.0rc1

2018-09-18 Thread Thananon Patinyasakdikul
Hi, I tested on our cluster (UTK). I will give a thumb up but I have some comments. What I understand with 4.0. - openib btl is disabled by default (can be turned on by mca) - pml ucx will be the default for infiniband hardware. - btl uct is for one-sided but can also be used for two sided as