Patches are always welcome. What would be great is a nice big warning that CMA
support is disabled because the processes are on different namespaces. Ideally
all MPI processes should be on the same namespace to ensure the best
performance.
-Nathan
> On Jul 21, 2019, at 2:53 PM, Adrian Reber v
For completeness I am mentioning my results also here.
To be able to mount file systems in the container it can only work if
user namespaces are used and even if the user IDs are all the same (in
each container and on the host), to be able to ptrace the kernel also
checks if the processes are in t
Gilles,
thanks again. Adding '--mca btl_vader_single_copy_mechanism none' helps
indeed.
The default seems to be 'cma' and that seems to use process_vm_readv()
and process_vm_writev(). That seems to require CAP_SYS_PTRACE, but
telling Podman to give the process CAP_SYS_PTRACE with '--cap-add=SYS_P
Adrian,
Can you try
mpirun --mca btl_vader_copy_mechanism none ...
Please double check the MCA parameter name, I am AFK
IIRC, the default copy mechanism used by vader directly accesses the remote
process address space, and this requires some permission (ptrace?) that might
be dropped by podman
So upstream Podman was really fast and merged a PR which makes my
wrapper unnecessary:
Add support for --env-host : https://github.com/containers/libpod/pull/3557
As commented in the PR I can now start mpirun with Podman without a
wrapper:
$ mpirun --hostfile ~/hosts --mca orte_tmpdir_base /tmp
Not really a relevant reply, however Nomad has task drivers for Docker and
Singularity
https://www.hashicorp.com/blog/singularity-and-hashicorp-nomad-a-perfect-fit
I'm not sure if it woul dbe easier to set up an MPI enviroment with Nomad
though
On Thu, 11 Jul 2019 at 11:08, Adrian Reber via users
Gilles,
thanks for pointing out the environment variables. I quickly created a
wrapper which tells Podman to re-export all OMPI_ and PMIX_ variables
(grep "\(PMIX\|OMPI\)"). Now it works:
$ mpirun --hostfile ~/hosts ./wrapper -v /tmp:/tmp --userns=keep-id --net=host
mpi-test /home/mpi/hello
He
Adrian,
the MPI application relies on some environment variables (they typically
start with OMPI_ and PMIX_).
The MPI application internally uses a PMIx client that must be able to
contact a PMIx server
(that is included in mpirun and the orted daemon(s) spawned on the
remote hosts).
lo
I did a quick test to see if I can use Podman in combination with Open
MPI:
[test@test1 ~]$ mpirun --hostfile ~/hosts podman run
quay.io/adrianreber/mpi-test /home/mpi/hello
Hello, world (1 procs total)
--> Process # 0 of 1 is alive. ->789b8fb622ef
Hello, world (1 procs total)
-->
