Re: [OMPI users] mpirun on Kubuntu 20.4.1 hangs

2020-10-21 Thread Jeff Squyres (jsquyres) via users 
There's huge differences between Open MPI v2.1.1 and v4.0.3 (i.e., years of 
development effort); it would be very hard to categorize them all; sorry!

What happens if you

mpirun -np 1 touch /tmp/foo

(Yes, you can run non-MPI apps through mpirun)

Is /tmp/foo created?  (i.e., did the job run, and mpirun is somehow not 
terminating)



On Oct 21, 2020, at 12:22 PM, Jorge SILVA via users 
mailto:users@lists.open-mpi.org>> wrote:


Hello Gus,

 Thank you for your answer..  Unfortunately my problem is much more basic. I  
didn't try to run the program in both computers , but just to run something in 
one computer. I just installed the new OS an openmpi in two different 
computers, in the standard way, with the same result.

For example:

In kubuntu20.4.1 LTS with openmpi 4.0.3-0ubuntu

jorge@gcp26:~/MPIRUN$ cat hello.f90
 print*,"Hello World!"
end
jorge@gcp26:~/MPIRUN$ mpif90 hello.f90 -o hello
jorge@gcp26:~/MPIRUN$ ./hello
 Hello World!
jorge@gcp26:~/MPIRUN$ mpirun -np 1 hello<---here  the program hangs with no 
output
^C^Cjorge@gcp26:~/MPIRUN$

The mpirun task sleeps with no output, and only twice ctrl-C ends the execution 
 :

jorge   5540  0.1  0.0  44768  8472 pts/8S+   17:54   0:00 mpirun -np 1 
hello


In kubuntu 18.04.5 LTS with openmpi 2.1.1, of course, the same program gives

jorge@gcp30:~/MPIRUN$ cat hello.f90
 print*, "Hello World!"
 END
jorge@gcp30:~/MPIRUN$ mpif90 hello.f90 -o hello
jorge@gcp30:~/MPIRUN$ ./hello
 Hello World!
jorge@gcp30:~/MPIRUN$ mpirun -np 1 hello
 Hello World
jorge@gcp30:~/MPIRUN$


Even just typing mpirun hangs without the usual error message.

Are there any changes between the two versions of openmpi that I miss?  Some 
package lacking to mpirun ?

Thank you again for your help

Jorge

Le 21/10/2020 à 00:20, Gus Correa a écrit :
Hi Jorge

You may have an active firewall protecting either computer or both,
and preventing mpirun to start the connection.
Your /etc/hosts file may also not have the computer IP addresses.
You may also want to try the --hostfile option.
Likewise, the --verbose option may also help diagnose the problem.

It would help if you send the mpirun command line, the hostfile (if any),
error message if any, etc.


These FAQs may help diagnose and solve the problem:

https://www.open-mpi.org/faq/?category=running#diagnose-multi-host-problems
https://www.open-mpi.org/faq/?category=running#mpirun-hostfile
https://www.open-mpi.org/faq/?category=running

I hope this helps,
Gus Correa

On Tue, Oct 20, 2020 at 4:47 PM Jorge SILVA via users 
mailto:users@lists.open-mpi.org>> wrote:
Hello,

I installed kubuntu20.4.1 with openmpi 4.0.3-0ubuntu in two different
computers in the standard way. Compiling with mpif90 works, but mpirun
hangs with no output in both systems. Even mpirun command without
parameters hangs and only twice ctrl-C typing can end the sleeping
program. Only  the command

 mpirun --help

gives the usual output.

Seems that is something related to the terminal output, but the command
worked well for Kubuntu 18.04. Is there a way to debug or fix this
problem (without re-compiling from sources, etc)? Is it a known problem?

Thanks,

  Jorge



--
Jeff Squyres
jsquy...@cisco.com

Re: [OMPI users] mpirun on Kubuntu 20.4.1 hangs

2020-10-21 Thread Jorge SILVA via users 

Hello Jeff,

The  program is not executed, seems waits for something to connect with 
(why twice ctrl-C ?)


jorge@gcp26:~/MPIRUN$ mpirun -np 1 touch /tmp/foo
^C^C

jorge@gcp26:~/MPIRUN$ ls -l /tmp/foo
ls: impossible d'accéder à '/tmp/foo': Aucun fichier ou dossier de ce type

no file  is created..

In fact, my question was if are there differences in mpirun usage  
between these versions..  The


mpirun -help

gives a different output as expected, but I  tried a lot of options 
without any success.



Le 21/10/2020 à 21:16, Jeff Squyres (jsquyres) a écrit :
There's huge differences between Open MPI v2.1.1 and v4.0.3 (i.e., 
years of development effort); it would be very hard to categorize them 
all; sorry!


What happens if you

    mpirun -np 1 touch /tmp/foo

(Yes, you can run non-MPI apps through mpirun)

Is /tmp/foo created?  (i.e., did the job run, and mpirun is somehow 
not terminating)




On Oct 21, 2020, at 12:22 PM, Jorge SILVA via users 
mailto:users@lists.open-mpi.org>> wrote:


Hello Gus,

 Thank you for your answer..  Unfortunately my problem is much more 
basic. I  didn't try to run the program in both computers , but just 
to run something in one computer. I just installed the new OS an 
openmpi in two different computers, in the standard way, with the 
same result.


For example:

In kubuntu20.4.1 LTS with openmpi 4.0.3-0ubuntu

jorge@gcp26:~/MPIRUN$ cat hello.f90
 print*,"Hello World!"
end
jorge@gcp26:~/MPIRUN$ mpif90 hello.f90 -o hello
jorge@gcp26:~/MPIRUN$ ./hello
 Hello World!
jorge@gcp26:~/MPIRUN$ mpirun -np 1 hello <---here  the program hangs 
with no output

^C^Cjorge@gcp26:~/MPIRUN$

The mpirun task sleeps with no output, and only twice ctrl-C ends the 
execution  :


jorge   5540  0.1  0.0 44768  8472 pts/8    S+   17:54   0:00 
mpirun -np 1 hello


In kubuntu 18.04.5 LTS with openmpi 2.1.1, of course, the same 
program gives


jorge@gcp30:~/MPIRUN$ cat hello.f90
 print*, "Hello World!"
 END
jorge@gcp30:~/MPIRUN$ mpif90 hello.f90 -o hello
jorge@gcp30:~/MPIRUN$ ./hello
 Hello World!
jorge@gcp30:~/MPIRUN$ mpirun -np 1 hello
 Hello World
jorge@gcp30:~/MPIRUN$


Even just typing mpirun hangs without the usual error message.

Are there any changes between the two versions of openmpi that I 
miss?  Some package lacking to mpirun ?


Thank you again for your help

Jorge


Le 21/10/2020 à 00:20, Gus Correa a écrit :

Hi Jorge

You may have an active firewall protecting either computer or both,
and preventing mpirun to start the connection.
Your /etc/hosts file may also not have the computer IP addresses.
You may also want to try the --hostfile option.
Likewise, the --verbose option may also help diagnose the problem.

It would help if you send the mpirun command line, the hostfile (if 
any),

error message if any, etc.


These FAQs may help diagnose and solve the problem:

https://www.open-mpi.org/faq/?category=running#diagnose-multi-host-problems
https://www.open-mpi.org/faq/?category=running#mpirun-hostfile
https://www.open-mpi.org/faq/?category=running

I hope this helps,
Gus Correa

On Tue, Oct 20, 2020 at 4:47 PM Jorge SILVA via users 
mailto:users@lists.open-mpi.org>> wrote:


Hello,

I installed kubuntu20.4.1 with openmpi 4.0.3-0ubuntu in two
different
computers in the standard way. Compiling with mpif90 works, but
mpirun
hangs with no output in both systems. Even mpirun command without
parameters hangs and only twice ctrl-C typing can end the sleeping
program. Only  the command

 mpirun --help

gives the usual output.

Seems that is something related to the terminal output, but the
command
worked well for Kubuntu 18.04. Is there a way to debug or fix this
problem (without re-compiling from sources, etc)? Is it a known
problem?

Thanks,

  Jorge




--
Jeff Squyres
jsquy...@cisco.com

[OMPI users] Delays in Open MPI mailing list

2020-10-21 Thread Jeff Squyres (jsquyres) via users 
FYI: We've been having some problems with our mailing list provider over the 
past few weeks.

No mails have been lost, but sometimes mails queue up endlessly at our mailing 
list provider until a human IT staffer goes in, fixes a problem, and 
effectively releases all the mails that have queued up.  This can result in 
large delays between when you send an email and when it is actually delivered 
to the list.

Sorry about this!

Our mailing list provider is working on it, and hopes to have it resolved soon 
(e.g., today they're rebuilding our mailman server from scratch in the hopes 
that it will be more reliable than the previous setup).

-- 
Jeff Squyres
jsquy...@cisco.com

Re: [OMPI users] mpirun on Kubuntu 20.4.1 hangs

2020-10-21 Thread Gilles Gouaillardet via users 
Hi Jorge,

If a firewall is running on your nodes, I suggest you disable it and try again

Cheers,

Gilles

On Wed, Oct 21, 2020 at 5:50 AM Jorge SILVA via users
 wrote:
>
> Hello,
>
> I installed kubuntu20.4.1 with openmpi 4.0.3-0ubuntu in two different
> computers in the standard way. Compiling with mpif90 works, but mpirun
> hangs with no output in both systems. Even mpirun command without
> parameters hangs and only twice ctrl-C typing can end the sleeping
> program. Only  the command
>
>  mpirun --help
>
> gives the usual output.
>
> Seems that is something related to the terminal output, but the command
> worked well for Kubuntu 18.04. Is there a way to debug or fix this
> problem (without re-compiling from sources, etc)? Is it a known problem?
>
> Thanks,
>
>   Jorge
>

Re: [OMPI users] Code failing when requesting all "processors"

2020-10-21 Thread Diego Zuccato via users 
Il 14/10/20 14:32, Jeff Squyres (jsquyres) ha scritto:

>> The version is 3.1.3 , as packaged in Debian Buster.
> The 3.1.x series is pretty old.  If you want to stay in the 3.1.x
> series, you might try upgrading to the latest -- 3.1.6.  That has a
> bunch of bug fixes compared to v3.1.3.
I'm bound to using distro packages...
I don't have the resources to also compile from sources and debug
interactions between different packages (OMPI, Slurm, OFED... just to
start, and every one would require an expert).

>> I don't know OpenMPI (or even MPI in general) much. Some time ago, I've
>> had to add a
>> mtl = psm2
>> line to /etc/openmpi/openmpi-mca-params.conf .
> This implies that you have Infinipath networking on your cluster.
Actually we have InfiniBand on most of the nodes. All Mellanox cards
(I've been warned about bad interactions between different vendors),
some ConnectX-3 cards (connected to a 40Gbps switch) and some ConnetX-5
ones (connected to a 100Gbps switch, linked to the first). The link
between the two switches is mostly unused, unless for the traffic to the
Gluster servers, over IPoIB.

> I can't imagine what installing gdb would do to mask the problem.  Strange.
Imagine my face when the program started working under gdb, then
continued even when launched directly with no binary changes... :)

-- 
Diego Zuccato
DIFA - Dip. di Fisica e Astronomia
Servizi Informatici
Alma Mater Studiorum - Università di Bologna
V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
tel.: +39 051 20 95786

[OMPI users] Is there a workaround available to get around fork() where I don't have access to the program's source code?

2020-10-21 Thread Ewen Chan via users 
To Whom It May Concern:

I have a program that I am running that has the MPI call fork() and when I run 
the program, I get the following warning:

--
An MPI process has executed an operation involving a call to the
"fork()" system call to create a child process.  Open MPI is currently
operating in a condition that could result in memory corruption or
other system errors; your MPI job may hang, crash, or produce silent
data corruption.  The use of fork() (or system() or other calls that
create child processes) is strongly discouraged.

The process that invoked fork was:

  Local host:  aes0 (PID 16225)
  MPI_COMM_WORLD rank: 0

If you are *absolutely sure* that your application will successfully
and correctly survive a call to fork(), you may disable this warning
by setting the mpi_warn_on_fork MCA parameter to 0.
--

I don't have access to the source code for this program and when I run it, I do 
get a segmentation fault.

Is there a way for me to work/get around this if I don't have access to the 
source code?

Thank you.

Sincerely,
Ewen

Re: [OMPI users] mpirun on Kubuntu 20.4.1 hangs

2020-10-21 Thread Jorge SILVA via users 

Hello Gus,

 Thank you for your answer..  Unfortunately my problem is much more 
basic. I  didn't try to run the program in both computers , but just to 
run something in one computer. I just installed the new OS an openmpi in 
two different computers, in the standard way, with the same result.


For example:

In kubuntu20.4.1 LTS with openmpi 4.0.3-0ubuntu

jorge@gcp26:~/MPIRUN$ cat hello.f90
 print*,"Hello World!"
end
jorge@gcp26:~/MPIRUN$ mpif90 hello.f90 -o hello
jorge@gcp26:~/MPIRUN$ ./hello
 Hello World!
jorge@gcp26:~/MPIRUN$ mpirun -np 1 hello <---here the program hangs with 
no output

^C^Cjorge@gcp26:~/MPIRUN$

The mpirun task sleeps with no output, and only twice ctrl-C ends the 
execution  :


jorge   5540  0.1  0.0  44768  8472 pts/8    S+   17:54 0:00 mpirun 
-np 1 hello


In kubuntu 18.04.5 LTS with openmpi 2.1.1, of course, the same program gives

jorge@gcp30:~/MPIRUN$ cat hello.f90
 print*, "Hello World!"
 END
jorge@gcp30:~/MPIRUN$ mpif90 hello.f90 -o hello
jorge@gcp30:~/MPIRUN$ ./hello
 Hello World!
jorge@gcp30:~/MPIRUN$ mpirun -np 1 hello
 Hello World
jorge@gcp30:~/MPIRUN$


Even just typing mpirun hangs without the usual error message.

Are there any changes between the two versions of openmpi that I miss?  
Some package lacking to mpirun ?


Thank you again for your help

Jorge


Le 21/10/2020 à 00:20, Gus Correa a écrit :

Hi Jorge

You may have an active firewall protecting either computer or both,
and preventing mpirun to start the connection.
Your /etc/hosts file may also not have the computer IP addresses.
You may also want to try the --hostfile option.
Likewise, the --verbose option may also help diagnose the problem.

It would help if you send the mpirun command line, the hostfile (if any),
error message if any, etc.


These FAQs may help diagnose and solve the problem:

https://www.open-mpi.org/faq/?category=running#diagnose-multi-host-problems
https://www.open-mpi.org/faq/?category=running#mpirun-hostfile
https://www.open-mpi.org/faq/?category=running

I hope this helps,
Gus Correa

On Tue, Oct 20, 2020 at 4:47 PM Jorge SILVA via users 
mailto:users@lists.open-mpi.org>> wrote:


Hello,

I installed kubuntu20.4.1 with openmpi 4.0.3-0ubuntu in two different
computers in the standard way. Compiling with mpif90 works, but
mpirun
hangs with no output in both systems. Even mpirun command without
parameters hangs and only twice ctrl-C typing can end the sleeping
program. Only  the command

 mpirun --help

gives the usual output.

Seems that is something related to the terminal output, but the
command
worked well for Kubuntu 18.04. Is there a way to debug or fix this
problem (without re-compiling from sources, etc)? Is it a known
problem?

Thanks,

  Jorge

Re: [OMPI users] mpirun on Kubuntu 20.4.1 hangs

2020-10-21 Thread Gus Correa via users 
Hi Jorge

You may have an active firewall protecting either computer or both,
and preventing mpirun to start the connection.
Your /etc/hosts file may also not have the computer IP addresses.
You may also want to try the --hostfile option.
Likewise, the --verbose option may also help diagnose the problem.

It would help if you send the mpirun command line, the hostfile (if any),
error message if any, etc.


These FAQs may help diagnose and solve the problem:

https://www.open-mpi.org/faq/?category=running#diagnose-multi-host-problems
https://www.open-mpi.org/faq/?category=running#mpirun-hostfile
https://www.open-mpi.org/faq/?category=running

I hope this helps,
Gus Correa

On Tue, Oct 20, 2020 at 4:47 PM Jorge SILVA via users <
users@lists.open-mpi.org> wrote:

> Hello,
>
> I installed kubuntu20.4.1 with openmpi 4.0.3-0ubuntu in two different
> computers in the standard way. Compiling with mpif90 works, but mpirun
> hangs with no output in both systems. Even mpirun command without
> parameters hangs and only twice ctrl-C typing can end the sleeping
> program. Only  the command
>
>  mpirun --help
>
> gives the usual output.
>
> Seems that is something related to the terminal output, but the command
> worked well for Kubuntu 18.04. Is there a way to debug or fix this
> problem (without re-compiling from sources, etc)? Is it a known problem?
>
> Thanks,
>
>   Jorge
>
>

Re: [OMPI users] Anyone try building openmpi 4.0.5 w/ llvm 11

2020-10-21 Thread SERGENT, MARC via users 
Hi Alan,

I do not have any experience building Open MPI with the new “flang” compiler 
from the LLVM project (previously known as f18), but I successfully built an 
Open MPI 4.0.5 with the legacy “flang” compiler on top of LLVM 9.0.0.
If your users’ requirement for LLVM is not strongly linked on LLVM 11.x 
versions, you can find tips on how to build this flang compiler here : 
https://github.com/flang-compiler/flang/wiki/Building-Flang

Regards,
Marc


Marc Sergent
HPC Runtime Systems Expert
BDS R HPC & Big Data SW
T: +33 (0) 4 76 29 81 31
Bull an Atos Company
1 Rue de Provence
38130 Echirolles, France

De : users  De la part de Alan Wild via users
Envoyé : vendredi 16 octobre 2020 01:54
À : users@lists.open-mpi.org
Cc : Alan Wild 
Objet : [OMPI users] Anyone try building openmpi 4.0.5 w/ llvm 11


Caution! External email. Do not open attachments or click links, unless this 
email comes from a known sender and you know the content is safe.
More specifically building the new “flang” compiler and compiling openmpi with 
the combination of clang/flang rather than clang/gfortran.

Configure is passing (including support for 16 byte REAL and COMPLEX types.  
However there is one file that uses REAL128 and CONPLE(REAL128) and I’m getting 
type in supported KIND=-1 errors.

(Quick aside I’m surprised that the one file is using the ISO_FORTRAN_ENV type 
names but configure is only checking for the non-standard REAL*# names.  Feels 
like an oversight to me.)

From what I’ve been able to find (and reading their iso_fortran_env.f90 module 
file the compiler should support the two types. I barely know enough FORTRAN to 
be dangerous (took one semester of F77 in like 1996) so I’m not really sure 
what I’m looking at here or what to try next.

I would really like to provide an openmpi build to my users that is a “pure 
LLVM” build.

-Alan
--
a...@madllama.net 
http://humbleville.blogspot.com