[jira] [Commented] (MESOS-5401) Add ability to inject a Volume of Nvidia GPU-related libraries into a docker container.
[
https://issues.apache.org/jira/browse/MESOS-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363734#comment-15363734
]
Benjamin Mahler commented on MESOS-5401:
{noformat}
commit 516d8ef0759937eb327893a01e2bdef8c7438b1c
Author: Kevin Klues
Date: Tue Jul 5 20:55:36 2016 -0700
Added test for using an Nvidia Docker image.
This test ensures that when using one of Nvidia's Docker images
(these contain a special label), we mount a volume that contains
the libraries and binaries.
Review: https://reviews.apache.org/r/49678/
{noformat}
> Add ability to inject a Volume of Nvidia GPU-related libraries into a docker
> container.
> ---
>
> Key: MESOS-5401
> URL: https://issues.apache.org/jira/browse/MESOS-5401
> Project: Mesos
> Issue Type: Improvement
>Reporter: Kevin Klues
>Assignee: Kevin Klues
> Labels: gpu, mesosphere
> Fix For: 1.0.0
>
>
> In order to support Nvidia GPUs with docker containers in Mesos, we need to
> be able to consolidate all Nvidia libraries into a common volume and inject
> that volume into the container.
> More info on why this is necessary here:
> https://github.com/NVIDIA/nvidia-docker/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5401) Add ability to inject a Volume of Nvidia GPU-related libraries into a docker container.
[
https://issues.apache.org/jira/browse/MESOS-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363605#comment-15363605
]
Benjamin Mahler commented on MESOS-5401:
{noformat}
commit 57b52a0ee4bc9016f6b3de29606015c1216fb5b1
Author: Kevin Klues
Date: Tue Jul 5 18:42:52 2016 -0700
Inject Nvidia libraries for Docker images in mesos containerizer.
When Docker images have the matching Nvidia label, we will inject
the volume which contains the Nvidia libraries / binaries.
Similar support will be added for the Docker containerizer.
Review: https://reviews.apache.org/r/49669/
{noformat}
> Add ability to inject a Volume of Nvidia GPU-related libraries into a docker
> container.
> ---
>
> Key: MESOS-5401
> URL: https://issues.apache.org/jira/browse/MESOS-5401
> Project: Mesos
> Issue Type: Improvement
>Reporter: Kevin Klues
>Assignee: Kevin Klues
> Labels: gpu, mesosphere
> Fix For: 1.0.0
>
>
> In order to support Nvidia GPUs with docker containers in Mesos, we need to
> be able to consolidate all Nvidia libraries into a common volume and inject
> that volume into the container.
> More info on why this is necessary here:
> https://github.com/NVIDIA/nvidia-docker/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5401) Add ability to inject a Volume of Nvidia GPU-related libraries into a docker container.
[ https://issues.apache.org/jira/browse/MESOS-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363514#comment-15363514 ] Kevin Klues commented on MESOS-5401: Last one of the patches in this patch set to enable this feature: https://reviews.apache.org/r/49669/ > Add ability to inject a Volume of Nvidia GPU-related libraries into a docker > container. > --- > > Key: MESOS-5401 > URL: https://issues.apache.org/jira/browse/MESOS-5401 > Project: Mesos > Issue Type: Improvement >Reporter: Kevin Klues >Assignee: Kevin Klues > Labels: gpu, mesosphere > Fix For: 1.0.0 > > > In order to support Nvidia GPUs with docker containers in Mesos, we need to > be able to consolidate all Nvidia libraries into a common volume and inject > that volume into the container. > More info on why this is necessary here: > https://github.com/NVIDIA/nvidia-docker/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5401) Add ability to inject a Volume of Nvidia GPU-related libraries into a docker container.
[
https://issues.apache.org/jira/browse/MESOS-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363425#comment-15363425
]
Benjamin Mahler commented on MESOS-5401:
{noformat}
commit a1423a5fe64c888003846b20b200f3142e6fca48
Author: Kevin Klues
Date: Tue Jul 5 15:22:35 2016 -0700
Implemented 'shouldInject()' in the 'NvidiaVolume' component.
We use the the 'com.nvidia.volumes.needed' label from
nvidia-docker to decide if we should inject the volume or not:
https://github.com/NVIDIA/nvidia-docker/wiki/Image-inspection
Review: https://reviews.apache.org/r/49615/
{noformat}
> Add ability to inject a Volume of Nvidia GPU-related libraries into a docker
> container.
> ---
>
> Key: MESOS-5401
> URL: https://issues.apache.org/jira/browse/MESOS-5401
> Project: Mesos
> Issue Type: Improvement
>Reporter: Kevin Klues
>Assignee: Kevin Klues
> Labels: gpu, mesosphere
> Fix For: 1.0.0
>
>
> In order to support Nvidia GPUs with docker containers in Mesos, we need to
> be able to consolidate all Nvidia libraries into a common volume and inject
> that volume into the container.
> More info on why this is necessary here:
> https://github.com/NVIDIA/nvidia-docker/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5401) Add ability to inject a Volume of Nvidia GPU-related libraries into a docker container.
[ https://issues.apache.org/jira/browse/MESOS-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15361796#comment-15361796 ] Kevin Klues commented on MESOS-5401: This patch adds support for deciding if we should inject a volume or not: https://reviews.apache.org/r/49615/ > Add ability to inject a Volume of Nvidia GPU-related libraries into a docker > container. > --- > > Key: MESOS-5401 > URL: https://issues.apache.org/jira/browse/MESOS-5401 > Project: Mesos > Issue Type: Improvement >Reporter: Kevin Klues >Assignee: Kevin Klues > Labels: gpu, mesosphere > Fix For: 1.0.0 > > > In order to support Nvidia GPUs with docker containers in Mesos, we need to > be able to consolidate all Nvidia libraries into a common volume and inject > that volume into the container. > More info on why this is necessary here: > https://github.com/NVIDIA/nvidia-docker/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5401) Add ability to inject a Volume of Nvidia GPU-related libraries into a docker container.
[
https://issues.apache.org/jira/browse/MESOS-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15360358#comment-15360358
]
Benjamin Mahler commented on MESOS-5401:
{noformat}
commit a7185a88c67eb462a0fe8da805193ae027fadca3
Author: Kevin Klues
Date: Sat Jul 2 15:41:05 2016 -0700
Added a new 'NvidiaVolume' component.
This component is responsible for building up a consolidated volume of
binaries / libraries from the user-space portion of the Nvidia GPU
drivers so that it can be injected into a container as a single
volume. Its core functionality mimics that of the
'nvidia-docker-plugin': https://github.com/NVIDIA/nvidia-docker/
We currently only implement the 'create()' function which is
responsible for building up the volume itself. In a subsequent commit
we will add support for reading a Docker image manifest and deciding
whether we should inject the volume into the docker container.
Review: https://reviews.apache.org/r/49565/
{noformat}
{noformat}
commit 14458c81ecdc0f592a6f07db551a237f7173eca7
Author: Kevin Klues
Date: Sat Jul 2 16:39:35 2016 -0700
Integrated the 'NvidiaVolume' component into 'NvidiaComponents'.
This makes it so that the 'NvidiaVolume' component can be shared
across containerizers in the same way that the 'NvidiaGpuAllocator' is
currently being shared.
Review: https://reviews.apache.org/r/49566/
{noformat}
> Add ability to inject a Volume of Nvidia GPU-related libraries into a docker
> container.
> ---
>
> Key: MESOS-5401
> URL: https://issues.apache.org/jira/browse/MESOS-5401
> Project: Mesos
> Issue Type: Improvement
>Reporter: Kevin Klues
>Assignee: Kevin Klues
> Labels: gpu, mesosphere
> Fix For: 1.0.0
>
>
> In order to support Nvidia GPUs with docker containers in Mesos, we need to
> be able to consolidate all Nvidia libraries into a common volume and inject
> that volume into the container.
> More info on why this is necessary here:
> https://github.com/NVIDIA/nvidia-docker/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5401) Add ability to inject a Volume of Nvidia GPU-related libraries into a docker container.
[ https://issues.apache.org/jira/browse/MESOS-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359195#comment-15359195 ] Kevin Klues commented on MESOS-5401: I am actively working on this now. You can track the progress on my private nvidia branch on github: https://github.com/klueska-mesosphere/mesos/commits/nvidia This link will likely be broken once this work is complete. > Add ability to inject a Volume of Nvidia GPU-related libraries into a docker > container. > --- > > Key: MESOS-5401 > URL: https://issues.apache.org/jira/browse/MESOS-5401 > Project: Mesos > Issue Type: Improvement >Reporter: Kevin Klues >Assignee: Kevin Klues > > In order to support Nvidia GPUs with docker containers in Mesos, we need to > be able to consolidate all Nvidia libraries into a common volume and inject > that volume into the container. > More info on why this is necessary here: > https://github.com/NVIDIA/nvidia-docker/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5401) Add ability to inject a Volume of Nvidia GPU-related libraries into a docker container.
[ https://issues.apache.org/jira/browse/MESOS-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358242#comment-15358242 ] Sunzhe commented on MESOS-5401: --- Oh, I see. What the progress is going on and would you like some help? I have got some idea. Since you have got so many things to do, if we work together, might make the progress faster. > Add ability to inject a Volume of Nvidia GPU-related libraries into a docker > container. > --- > > Key: MESOS-5401 > URL: https://issues.apache.org/jira/browse/MESOS-5401 > Project: Mesos > Issue Type: Improvement >Reporter: Kevin Klues >Assignee: Kevin Klues > > In order to support Nvidia GPUs with docker containers in Mesos, we need to > be able to consolidate all Nvidia libraries into a common volume and inject > that volume into the container. > More info on why this is necessary here: > https://github.com/NVIDIA/nvidia-docker/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5401) Add ability to inject a Volume of Nvidia GPU-related libraries into a docker container.
[ https://issues.apache.org/jira/browse/MESOS-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357288#comment-15357288 ] Kevin Klues commented on MESOS-5401: The crux of the issue is that we need to be able consolidate all of the Nvidia binaries/libraries into a single volume that we inject into a docker container. The goal is to let users leverage the nvidia Docker images (https://hub.docker.com/r/nvidia/) without any added effort on their behalf. Using docker they are able to launch containers from these images by simply running `nvidia-docker run ...` (i.e. they are unaware that a magic volume is being injected on their behalf). On Mesos we want the experience to be similar. The information at the link below should make things a bit more clear as to why this is necessary: https://github.com/NVIDIA/nvidia-docker/wiki/NVIDIA-driver. We originally planned on building this functionality as an isolator module, but there some some limitations with the current isolator interface that prohibit us from doing this properly. Building it as an isolator module would also mean that it couldn't be shared by the docker containerizer (which we plan to add support for in the future). > Add ability to inject a Volume of Nvidia GPU-related libraries into a docker > container. > --- > > Key: MESOS-5401 > URL: https://issues.apache.org/jira/browse/MESOS-5401 > Project: Mesos > Issue Type: Improvement >Reporter: Kevin Klues >Assignee: Kevin Klues > > In order to support Nvidia GPUs with docker containers in Mesos, we need to > be able to consolidate all Nvidia libraries into a common volume and inject > that volume into the container. > More info on why this is necessary here: > https://github.com/NVIDIA/nvidia-docker/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5401) Add ability to inject a Volume of Nvidia GPU-related libraries into a docker container.
[ https://issues.apache.org/jira/browse/MESOS-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356662#comment-15356662 ] Sunzhe commented on MESOS-5401: --- This issue you mean, take the Nvidia GPU-related libraries as a Volume and Mount to a docker container, is it? > Add ability to inject a Volume of Nvidia GPU-related libraries into a docker > container. > --- > > Key: MESOS-5401 > URL: https://issues.apache.org/jira/browse/MESOS-5401 > Project: Mesos > Issue Type: Improvement >Reporter: Kevin Klues >Assignee: Kevin Klues > > In order to support Nvidia GPUs with docker containers in Mesos, we need to > be able to consolidate all Nvidia libraries into a common volume and inject > that volume into the container. > More info on why this is necessary here: > https://github.com/NVIDIA/nvidia-docker/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
