I'd looked into doing this some time ago- there's been some desire for this expressed within our community and I can see some value in supporting docker in our environment.
Ultimately we had to say no because of the security issues you've indicated. We're hoping to provide some resources for prototyping with docker (both using 3rd party containers and developing our own), but ultimately we decided that if it was worth running on our cluster, it was worth porting the docker application into our environment (via modules or similar). That said, it might be possible to safely start a container for the end user if there was some mechanism for starting a container in a sanitized mode: run in the foreground, no privileges, no devices, no data volumes, a standard networking configuration, etc. Since slurmd runs as root, it would have the necessary privileges... whether slurmd would be able to manage the container properly is another question. On Tue, May 19, 2015 at 10:14 AM, Kilian Cavalotti < [email protected]> wrote: > > On Tue, May 19, 2015 at 9:28 AM, Michael Jennings <[email protected]> wrote: > > What Chris is asking for, I *think*, is what we're looking for as well > -- anyone who has figured out a way to allow users to execute jobs inside > user-supplied (or at least user-specified) Docker containers. It would be > nice to be able to allow users to supply not only the data, scripts, and > programs that compose their job but also the OS environment (in the form of > a Docker container or Dockerfile) within which it should execute. > > One major downside to running Docker containers in a shared HPC > cluster (to me at least), is that the default user in a container is > root. And that it can easily map and access the host filesystem from > inside the container. Letting users run as root on a shared cluster is > a major no-go from my perspective. So until Docker folks figure out a > way to avoid this (and work on this seems to have just started very > recently: https://github.com/docker/docker/issues/12949), I don't see > much appeal from running Docker containers on a shared HPC cluster. > There may be other use cases, of course. > > But if users running as root is not an issue, what more is needed from > Slurm to launch containers? I may very well be missing something, but > If you have a docker daemon running on all of your compute nodes, and > provided users can access the docker socket/port, they can submit jobs > that call "docker run", can't they? > > Cheers, > -- > Kilian
