Hi there,
I'm planning to build new docker images in yade's gitlab for production,
and possibly for development (see second part of this message, some
background comes first). This is open to suggestions.
* Background:
I recently started playing with "Singularity" images since I found our
HPC department made it available on the clusters. There was also a user
mentioning that on launchpad recently. From end-user POV, singularity
images work like docker images, but a very practical difference is that
it is allowed on our (and others') HPC. Docker isn't, for security reason.
It made running yade so easy. The above command worked immediately, and
should work just the same on every system with singularity installed:
/ssh myHPC//
//singularity exec
docker://registry.gitlab.com/bchareyre/docker-yade:ubuntu20.04-daily
yadedaily --check/
or equivalently: /
///export YADE=//'singularity exec
docker://registry.gitlab.com/bchareyre/docker-yade:ubuntu20.04-daily
yadedaily'
$YADE --check
$YADE myScript.py
$ etc.
//
Key points:
1- singularity accepts docker images in input.
2- the above command is using some custom docker with yadedaily
pre-installed (which then needs to be downloadable from somewhere where
docker is permitted)
3- it is compatible with MPI(!). The host system's MPI is able to
communicate with the image system's MPI in a scenario like this, as if
it was just yade running natively on the host:
/mpirun -np 500 $YADE someParrallelStuff.py
/4- a condition for this MPI magic to work is that the mpi library is in
the same version for the host and for the executed image
5- performance: no measurable difference compared to a yade compiled on
the host (be it running -j1, -jN or mpiexec).
For the moment the custom dockers are built in [1]
<https://gitlab.com/bchareyre/docker-yade>.
I'm also building a Singularity images with [2]
<https://gitlab.com/bchareyre/yade-singularity/-/blob/master/.gitlab-ci.yml>
but I didn't really use it since I can build it from docker directly on
the cluster (building the singularity image is implicit in /singularity
exec docker://.../). Building on-site may not be allowed everywhere,
though, and in that case [2] could be useful.
* What can be done:
I will move [1,2] or something similar to gitlab/yade-dev and advertise
it in the install page. Also build more versions for people to use them.
More versions because of the MPI point above (4): depending on the host
system someone may want OMPI v1 (unbuntu16), or v2 (ubuntu18), etc.
For production images it would make sense to just use canonical
debian/ubuntu with yade and/or yadedaily preinstalled. But, it is not
exactly what I did for the moment. Instead I used docker images from our
registry. Which implies the images have yade, and also what it needs to
compile yade (I didn't test compilation yet but it should work).
I was thinking of splitting that into two types of images; minimal
images for production and "dev" images with all compilation
pre-requisites. Then I realized that the best "dev" image would be - by
far - one reflecting the state of the system at the end of our current
pipeline, i.e. one with a full /build folder and possibly ccache info
(if not too large).
If such dev images were pushed to yade registry then anyone could grab
latest build and recompile incrementally. It could save a lot of
(compilation) time for us when trying to debug something on multiple
distros.
And what about that?: compiling with a ubuntu20 docker image on a
ubuntu20 host should make it possible to use the pipeline's ccache while
still running yade on the native system (provided that the install path
is in the host filesystem).
Maybe pushing to registry could be done directly as part of current
pipeline, not sure yet. I am still thinking about some aspects but I
think you get the general idea. Suggestions and advices are welcome. :)
Chers
Bruno
[1] https://gitlab.com/bchareyre/docker-yade
[2]
https://gitlab.com/bchareyre/yade-singularity/-/blob/master/.gitlab-ci.yml
/
/
/
/
/
/
/
/
_______________________________________________
Mailing list: https://launchpad.net/~yade-dev
Post to : yade-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yade-dev
More help : https://help.launchpad.net/ListHelp