Re: [OMPI users] MPI Exit Code:1 on an OpenFoam application

2021-01-10 Thread Tony Ladd via users
Kahnbein Kai via users wrote: [External Email] Hey Tony, it works without the -parallel flag, all four cpu's are at 100% and running fine. Best regards Kai Am 05.01.21 um 20:36 schrieb Tony Ladd via users: Just run the executable without mpirun and the -parallel flag. On 1/2/21 11:39 PM,

Re: [OMPI users] MPI Exit Code:1 on an OpenFoam application

2021-01-05 Thread Tony Ladd via users
Just run the executable without mpirun and the -parallel flag. On 1/2/21 11:39 PM, Kahnbein Kai via users wrote: *[External Email]* Ok, sorry, what do you mean with the "serial version" ? Best regards Kai Am 31.12.20 um 16:25 schrieb tladd via users: I did not see the whole email chain befo

Re: [OMPI users] Problem in starting openmpi job - no output just hangs - SOLVED

2020-09-01 Thread Tony Ladd via users
Jeff I found the solution - rdma needs significant memory so the limits on the shell have to be increased. I needed to add the lines * soft memlock unlimited * hard memlock unlimited to the end of the file /etc/security/limits.conf. After that the openib driver loads and everything is fine -

Re: [OMPI users] Problem in starting openmpi job - no output just hangs

2020-08-24 Thread Tony Ladd via users
f [older] hardware (including UCX support on that hardware). But be aware that openib is definitely going away; it is wholly being replaced by UCX. It may be that your only option is to stick with older software stacks in these hardware environments. On Aug 23, 2020, at 9:46 PM, Tony Ladd vi

Re: [OMPI users] Problem in starting openmpi job - no output just hangs

2020-08-23 Thread Tony Ladd via users
nfo on one node ibdiagnet on one node On Sun, 23 Aug 2020 at 05:02, Tony Ladd via users mailto:users@lists.open-mpi.org>> wrote: Hi Jeff I installed ucx as you suggested. But I can't get even the simplest code (ucp_client_server) to work across the network. I can comp

Re: [OMPI users] Problem in starting openmpi job - no output just hangs

2020-08-22 Thread Tony Ladd via users
Jeff Squyres (jsquyres) wrote: [External Email] Tony -- Have you tried compiling Open MPI with UCX support? This is Mellanox (NVIDIA's) preferred mechanism for InfiniBand support these days -- the openib BTL is legacy. You can run: mpirun --mca pml ucx ... On Aug 19, 2020, at 12:46 PM,

Re: [OMPI users] Problem in starting openmpi job - no output just hangs

2020-08-19 Thread Tony Ladd via users
One other update. I compiled OpenMPI-4.0.4 The outcome was the same but there is no mention of ibv_obj this time. Tony -- Tony Ladd Chemical Engineering Department University of Florida Gainesville, Florida 32611-6005 USA Email: tladd-"(AT)"-che.ufl.edu Webhttp://ladd.che.ufl.edu Tel:

Re: [OMPI users] Problem in starting openmpi job - no output just hangs

2020-08-17 Thread Tony Ladd via users
My apologies - I did not read the FAQ's carefully enough - with regard to 14: 1. openib 2. Ubuntu supplied drivers etc. 3. Ubuntu 18.04  4.15.0-112-generic 4. opensm-3.3.5_mlnx-0.1.g6b18e73 5. Attached 6. Attached 7. unlimited on foam and 16384 on f34 I changed the ulimit to unlimited on

[OMPI users] Problem in starting openmpi job - no output just hangs

2020-08-17 Thread Tony Ladd via users
I would very much appreciate some advice in how to debug this problem. I am trying to get OpenMPI to work on my reconfigured cluster - upgrading from Centos 5 to Ubuntu 18. The problem is that a simple job using Intel's IMB message passing test code will not run on any of the new clients (4 so