What interconnect are you using at run time? That is, are you using Ethernet or InfiniBand or Omnipath?
Sent from my iPad On Mar 4, 2021, at 5:05 AM, Raut, S Biplab via users <users@lists.open-mpi.org> wrote: [AMD Official Use Only - Internal Distribution Only] After downloading a particular openMPI version, let’s say v3.1.1 from https://download.open-mpi.org/release/open-mpi/v3.1/openmpi-3.1.1.tar.gz<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownload.open-mpi.org%2Frelease%2Fopen-mpi%2Fv3.1%2Fopenmpi-3.1.1.tar.gz&data=04%7C01%7Cmichael.william.heinz%40cornelisnetworks.com%7C8f8e82e9272f46b08a1608d8def4fbb9%7C4dbdb7da74ee4b458747ef5ce5ebe68a%7C0%7C0%7C637504491099833683%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=A7ia%2FTfeTEntAcNxJ%2FCaDLVoBVlHFcMkMojFM3iVwBo%3D&reserved=0> , I follow the below steps. ./configure --prefix="$INSTALL_DIR" --enable-mpi-fortran --enable-mpi-cxx --enable-shared=yes --enable-static=yes --enable-mpi1-compatibility make -j make install export PATH=$INSTALL_DIR/bin:$PATH export LD_LIBRARY_PATH=$INSTALL_DIR/lib:$LD_LIBRARY_PATH Additionally, I also install libnuma-dev on the machine. For all the machines having Ubuntu 18.04 and 19.04, it works correctly and results in expected performance/GFLOPS. But, when OS is changed to Ubuntu 20.04, then I start getting the issues as mentioned in my original/previous mail below. With Regards, S. Biplab Raut From: users <users-boun...@lists.open-mpi.org> On Behalf Of John Hearns via users Sent: Thursday, March 4, 2021 1:53 PM To: Open MPI Users <users@lists.open-mpi.org> Cc: John Hearns <hear...@gmail.com> Subject: Re: [OMPI users] Stable and performant openMPI version for Ubuntu20.04 ? [CAUTION: External Email] How are you installing the OpenMPI versions? Are you using packages which are distributed by the OS? It might be worth looking at using Easybuid or Spack https://docs.easybuild.io/en/latest/Introduction.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.easybuild.io%2Fen%2Flatest%2FIntroduction.html&data=04%7C01%7Cmichael.william.heinz%40cornelisnetworks.com%7C8f8e82e9272f46b08a1608d8def4fbb9%7C4dbdb7da74ee4b458747ef5ce5ebe68a%7C0%7C0%7C637504491099833683%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XDC7EPzYb23xFekaxFmMWFPLVi%2FiIzyQpjnWRJP9enA%3D&reserved=0> https://spack.readthedocs.io/en/latest/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspack.readthedocs.io%2Fen%2Flatest%2F&data=04%7C01%7Cmichael.william.heinz%40cornelisnetworks.com%7C8f8e82e9272f46b08a1608d8def4fbb9%7C4dbdb7da74ee4b458747ef5ce5ebe68a%7C0%7C0%7C637504491099843640%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Pxgm8BVqHiakTEh2q68slFcMDJxK1otfmTyWFUOMUEY%3D&reserved=0> On Thu, 4 Mar 2021 at 07:35, Raut, S Biplab via users <users@lists.open-mpi.org<mailto:users@lists.open-mpi.org>> wrote: [AMD Official Use Only - Internal Distribution Only] Dear Experts, Until recently, I was using openMPI3.1.1 to run single node 128 ranks MPI application on Ubuntu18.04 and Ubuntu19.04. But, now the OS on these machines are upgraded to Ubuntu20.04, and I have been observing program hangs with openMPI3.1.1 version. So, I tried with openMPI4.0.5 version – The program ran properly without any issues but there is a performance regression in my application. Can I know the stable openMPI version recommended for Ubuntu20.04 that has no known regression compared to v3.1.1. With Regards, S. Biplab Raut
