Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-12 Thread Bertini, Denis Dr. via users
From: George Bosilca Sent: Saturday, February 12, 2022 7:38:02 AM To: Bertini, Denis Dr. Cc: Open MPI Users; Joseph Schuchart Subject: Re: [OMPI users] Using OSU benchmarks for checking Infiniband network I am not sure I understand the comment about MPI_T. Each network card has internal

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-11 Thread George Bosilca via users
11, 2022 4:25:42 PM > *To:* Open MPI Users > *Cc:* Joseph Schuchart; Bertini, Denis Dr. > *Subject:* Re: [OMPI users] Using OSU benchmarks for checking Infiniband > network > > Collecting data during execution is possible in OMPI either with an > external tool, such as mpiP, or the in

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-11 Thread Bertini, Denis Dr. via users
From: George Bosilca Sent: Friday, February 11, 2022 4:25:42 PM To: Open MPI Users Cc: Joseph Schuchart; Bertini, Denis Dr. Subject: Re: [OMPI users] Using OSU benchmarks for checking Infiniband network Collecting data during execution is possible in OMPI either

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-11 Thread George Bosilca via users
uot; > > But i do not know what it is meant with "modified" openMPI ? > > > Cheers, > > Denis > > > -- > *From:* Joseph Schuchart > *Sent:* Friday, February 11, 2022 3:02:36 PM > *To:* Bertini, Denis Dr.; Open MPI Users > *Subje

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-11 Thread Bertini, Denis Dr. via users
I ? Cheers, Denis From: Joseph Schuchart Sent: Friday, February 11, 2022 3:02:36 PM To: Bertini, Denis Dr.; Open MPI Users Subject: Re: [OMPI users] Using OSU benchmarks for checking Infiniband network I am not aware of anything similar in Open MPI. Maybe OSU-INAM can

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-11 Thread Joseph Schuchart via users
@lists.open-mpi.org *Cc:* Joseph Schuchart *Subject:* Re: [OMPI users] Using OSU benchmarks for checking Infiniband network Hi Denis, Sorry if I missed it in your previous messages but could you also try running a different MPI implementation (MVAPICH) to see whether Open MPI is at fault

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-11 Thread Bertini, Denis Dr. via users
users Sent: Tuesday, February 8, 2022 4:02:53 PM To: users@lists.open-mpi.org Cc: Joseph Schuchart Subject: Re: [OMPI users] Using OSU benchmarks for checking Infiniband network Hi Denis, Sorry if I missed it in your previous messages but could you also try running a different MPI implementation

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-08 Thread Bertini, Denis Dr. via users
:02:53 PM To: users@lists.open-mpi.org Cc: Joseph Schuchart Subject: Re: [OMPI users] Using OSU benchmarks for checking Infiniband network Hi Denis, Sorry if I missed it in your previous messages but could you also try running a different MPI implementation (MVAPICH) to see whether Open MPI

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-08 Thread Joseph Schuchart via users
*From:* users on behalf of Gus Correa via users *Sent:* Monday, February 7, 2022 9:14:19 PM *To:* Open MPI Users *Cc:* Gus Correa *Subject:* Re: [OMPI users] Using OSU benchmarks for checking Infiniband network This may have changed

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-08 Thread Benson Muite via users
*Sent:* Monday, February 7, 2022 9:14:19 PM *To:* Open MPI Users *Cc:* Gus Correa *Subject:* Re: [OMPI users] Using OSU benchmarks for checking Infiniband network This may have changed since, but these used to be relevant points. Overall, the Open MPI FAQ have lots of good suggestions: https://ww

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-08 Thread Bertini, Denis Dr. via users
Do also check whether there are any performance measurements available from your infiniband switch provider that will allow checking of correct functionality at the single switch level. > > > Best > > Denis > > ------------------------ > *From:* users on behal

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-08 Thread Benson Muite via users
e switch level. Best Denis *From:* users on behalf of Gus Correa via users *Sent:* Monday, February 7, 2022 9:14:19 PM *To:* Open MPI Users *Cc:* Gus Correa *Subject:* Re: [OMPI users] Using OSU benchmarks for checking Infiniband network This may have changed since, but these used to be rele

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-08 Thread Bertini, Denis Dr. via users
Denis From: users on behalf of Gus Correa via users Sent: Monday, February 7, 2022 9:14:19 PM To: Open MPI Users Cc: Gus Correa Subject: Re: [OMPI users] Using OSU benchmarks for checking Infiniband network This may have changed since, but these used to be relevant

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-07 Thread Gus Correa via users
run mpi in verbose mode in order > > > > to further investigate this behaviour? > > > > Best > > > > Denis > > > > ---- > > *From:* users on behalf of Benson > > Muite via u

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-07 Thread Bertini, Denis Dr. via users
- > *From:* users on behalf of Benson > Muite via users > *Sent:* Monday, February 7, 2022 2:27:34 PM > *To:* users@lists.open-mpi.org > *Cc:* Benson Muite > *Subject:* Re: [OMPI users] Using OSU benchmarks for checking Infiniband > network > Hi, > Do you get sim

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-07 Thread Bertini, Denis Dr. via users
rs on behalf of Benson > Muite via users > *Sent:* Monday, February 7, 2022 2:27:34 PM > *To:* users@lists.open-mpi.org > *Cc:* Benson Muite > *Subject:* Re: [OMPI users] Using OSU benchmarks for checking Infiniband > network > Hi, > Do you get similar results when you rep

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-07 Thread Benson Muite via users
crepancy at the message size of 16384. May be there is a way to run mpi in verbose mode in order to further investigate this behaviour? Best Denis *From:* users on behalf of Benson Muite via users *Sent:* Monday, Febru

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-07 Thread Bertini, Denis Dr. via users
, February 7, 2022 2:27:34 PM To: users@lists.open-mpi.org Cc: Benson Muite Subject: Re: [OMPI users] Using OSU benchmarks for checking Infiniband network Hi, Do you get similar results when you repeat the test? Another job could have interfered with your run. Benson On 2/7/22 3:56 PM, Bertini, Denis Dr

Re: [OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-07 Thread Benson Muite via users
Hi, Do you get similar results when you repeat the test? Another job could have interfered with your run. Benson On 2/7/22 3:56 PM, Bertini, Denis Dr. via users wrote: Hi I am using OSU microbenchmarks compiled with openMPI 3.1.6 in order to check/benchmark the infiniband network for our

[OMPI users] Using OSU benchmarks for checking Infiniband network

2022-02-07 Thread Bertini, Denis Dr. via users
Hi I am using OSU microbenchmarks compiled with openMPI 3.1.6 in order to check/benchmark the infiniband network for our cluster. For that i use the collective all_reduce benchmark and run over 200 nodes, using 1 process per node. And this is the results i obtained 