Gilles,
thanks for pointing out the environment variables. I quickly created a
wrapper which tells Podman to re-export all OMPI_ and PMIX_ variables
(grep "\(PMIX\|OMPI\)"). Now it works:
$ mpirun --hostfile ~/hosts ./wrapper -v /tmp:/tmp --userns=keep-id --net=host
mpi-test /home/mpi/hello
Hello, world (2 procs total)
--> Process # 0 of 2 is alive. ->test1
--> Process # 1 of 2 is alive. ->test2
I need to tell Podman to mount /tmp from the host into the container, as
I am running rootless I also need to tell Podman to use the same user ID
in the container as outside (so that the Open MPI files in /tmp) can be
shared and I am also running without a network namespace.
So this is now with the full Podman provided isolation except the
network namespace. Thanks for you help!
Adrian
On Thu, Jul 11, 2019 at 04:47:21PM +0900, Gilles Gouaillardet via users wrote:
> Adrian,
>
>
> the MPI application relies on some environment variables (they typically
> start with OMPI_ and PMIX_).
>
> The MPI application internally uses a PMIx client that must be able to
> contact a PMIx server
>
> (that is included in mpirun and the orted daemon(s) spawned on the remote
> hosts).
>
> located on the same host.
>
>
> If podman provides some isolation between the app inside the container (e.g.
> /home/mpi/hello)
>
> and the outside world (e.g. mpirun/orted), that won't be an easy ride.
>
>
> Cheers,
>
>
> Gilles
>
>
> On 7/11/2019 4:35 PM, Adrian Reber via users wrote:
> > I did a quick test to see if I can use Podman in combination with Open
> > MPI:
> >
> > [test@test1 ~]$ mpirun --hostfile ~/hosts podman run
> > quay.io/adrianreber/mpi-test /home/mpi/hello
> >
> > Hello, world (1 procs total)
> > --> Process # 0 of 1 is alive. ->789b8fb622ef
> >
> > Hello, world (1 procs total)
> > --> Process # 0 of 1 is alive. ->749eb4e1c01a
> >
> > The test program (hello) is taken from
> > https://raw.githubusercontent.com/openhpc/ohpc/obs/OpenHPC_1.3.8_Factory/tests/mpi/hello.c
> >
> >
> > The problem with this is that each process thinks it is process 0 of 1
> > instead of
> >
> > Hello, world (2 procs total)
> > --> Process # 1 of 2 is alive. ->test1
> > --> Process # 0 of 2 is alive. ->test2
> >
> > My questions is how is the rank determined? What resources do I need to have
> > in my container to correctly determine the rank.
> >
> > This is Podman 1.4.2 and Open MPI 4.0.1.
> >
> > Adrian
> > _______________________________________________
> > users mailing list
> > users@lists.open-mpi.org
> > https://lists.open-mpi.org/mailman/listinfo/users
> >
> _______________________________________________
> users mailing list
> users@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/users
_______________________________________________
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users
