RE: GPU without root?

[Milovanov, Victor]

I was able to get the desired result by simply passing --resources=’gpus:1’

// Victor

From: Milovanov, Victor
Sent: Monday, August 7, 2017 3:03 PM
To: user@mesos.apache.org
Subject: GPU without root?

Hi!

I’d like to know if it is possible to use GPU resources with Mesos agent 
without running it as root?

The official documentation states, that isolation must be enabled for at least 
gpu/NVidia, which currently requires root. However, my scenario does not need 
isolation.

// Victor

[Design Doc] Standalone Container API

[Joseph Wu]

As part of work to improve storage support in Mesos [1], we will be adding
the ability to launch containers via the Mesos Containerizer, without going
through the traditional method (i.e. framework -> offer cycle -> launch
executor/task -> status updates -> etc).  Below I've linked a short design
document for interacting with these "standalone" containers:

https://docs.google.com/document/d/1DZVfZAOLtqd8kbiWHD4j29LzaYcNCh1k6QQnbggyTio/

Please feel free to comment on the doc (or on this thread) if you have any
comments or suggestions! Thanks!

[1]
https://lists.apache.org/thread.html/02871cb51ce6d0bec24770bcaaba07b52dcda0cdb87cbdd0871b82d1@%3Cdev.mesos.apache.org%3E

GPU without root?

[Milovanov, Victor]

Hi!

I’d like to know if it is possible to use GPU resources with Mesos agent 
without running it as root?

The official documentation states, that isolation must be enabled for at least 
gpu/NVidia, which currently requires root. However, my scenario does not need 
isolation.

// Victor

Re: Custom isolators - External container

[Thodoris Zois]

Hello, 

Thank you for your response, i will use the Mesos containerizer and run docker 
images as tasks, so i will use cgroups/cpu and cgroups/mem for isolation in 
order to get some results so i can compare them when in case i use my own 
isolators. Isolation module will work right? Because of the use of mesos 
containerizer..

Also, where i can find some workloads that are used along with mesos so i can 
evaluate better my results or traces so i can make my own workloads. What kind 
of tasks most people run with Mesos? 






> On 7 Aug 2017, at 21:16, Joseph Wu  wrote:
> 
> First off, the external containerizer was officially removed in Mesos 1.1.0 
> (it had been deprecated long before that release):
> https://issues.apache.org/jira/browse/MESOS-3370
> 
> ---
> 
> If you want to develop/deploy a new isolation method for Mesos, you should 
> first consider writing isolator modules (Mesos modules):
> https://github.com/apache/mesos/blob/master/include/mesos/slave/isolator.hpp
> 
> Isolator modules are only applicable for the Mesos containerizer, so if you 
> plan to run docker workloads, you can consider using built-in isolators 
> ("docker/runtime") that support running docker images in the Mesos 
> containerizer.
> 
> If you plan to use the Docker containerizer, your only choice is to develop a 
> custom executor to isolate tasks only within the same executor (docker will 
> take over isolating executors from each other).
> 
> ---
> 
> There are few benefits from running the Mesos agent inside a Docker container 
> and many pitfalls, so this practice is highly discouraged.  Instead, we 
> recommend running the Mesos agent directly via a supervisor (upstart, 
> systemd, etc.).  The agent itself is not containerized when run normally.
> 
>> On Sun, Aug 6, 2017 at 4:32 PM, Thodoris Zois  wrote:
>> Hello,
>> 
>> Is support of external containerizer removed from Mesos? Also, i have 
>> developed some isolators that i would like to use with Mesos. I found 3 ways 
>> to do that but i don't know what is the proper way and what are the 
>> advantages and disadvantages in each case.
>> 
>> The 1st one is as a Mesos module
>> 
>> The 2nd one is a custom executor
>> 
>> The 3rd one is the container image on agent.
>> 
>> What i am trying to do is to isolate docker tasks (images - one task per 
>> docker container) that run under the same agent with my own isolators.
>> 
>> What are the benefits of running agent in a big docker container and inside 
>> small docker containers as tasks?  If you don't run the agent under a big 
>> docker container  then by default is running under Mesos container while 
>> inside are running small docker containers with tasks? (Assume
>> that we don't run tasks under mesos container)
>> 
>> 
>> Thank you and sorry for the so many questions!
>> Thodoris
> 

Re: Custom isolators - External container

[Joseph Wu]

First off, the external containerizer was officially removed in Mesos 1.1.0
(it had been deprecated long before that release):
https://issues.apache.org/jira/browse/MESOS-3370

---

If you want to develop/deploy a new isolation method for Mesos, you should
first consider writing isolator modules (Mesos modules):
https://github.com/apache/mesos/blob/master/include/mesos/slave/isolator.hpp

Isolator modules are only applicable for the Mesos containerizer, so if you
plan to run docker workloads, you can consider using built-in isolators
("docker/runtime") that support running docker images in the Mesos
containerizer.

If you plan to use the Docker containerizer, your only choice is to develop
a custom executor to isolate tasks only within the same executor (docker
will take over isolating executors from each other).

---

There are few benefits from running the Mesos agent inside a Docker
container and many pitfalls, so this practice is highly discouraged.
Instead, we recommend running the Mesos agent directly via a supervisor
(upstart, systemd, etc.).  The agent itself is not containerized when run
normally.

On Sun, Aug 6, 2017 at 4:32 PM, Thodoris Zois  wrote:

> Hello,
>
> Is support of external containerizer removed from Mesos? Also, i have
> developed some isolators that i would like to use with Mesos. I found 3
> ways to do that but i don't know what is the proper way and what are the
> advantages and disadvantages in each case.
>
> The 1st one is as a Mesos module
>
> The 2nd one is a custom executor
>
> The 3rd one is the container image on agent.
>
> What i am trying to do is to isolate docker tasks (images - one task per
> docker container) that run under the same agent with my own isolators.
>
> What are the benefits of running agent in a big docker container and
> inside small docker containers as tasks?  If you don't run the agent under
> a big docker container  then by default is running under Mesos container
> while inside are running small docker containers with tasks? (Assume
> that we don't run tasks under mesos container)
>
>
> Thank you and sorry for the so many questions!
> Thodoris
>

Re: Command Executor

[James Peach]

> On Aug 5, 2017, at 3:03 AM, Oeg Bizz  wrote:
> 
> I have a framework that relies on information sent by a custom Java Command 
> Executor; think of some sort of heartbeat.  I start getting hearbeats after I 
> send a task to that mesos-slave, but never before that.  That makes me assume 
> that the CommandExecutor is not started until a task is submitted to be 
> executed by that agent.  Is there a way to tell mesos-slave to start the 
> ComandExecutor as soon as it starts running?

Not AFAIK. Executors are always spawned in order to execute tasks. In your 
case, what is the heartbeat for, if there are no tasks on the agent?

J

Re: Command Executor

[Oeg Bizz]

I sent this over the weekend so I can see how it would have been missed, does 
anyone have a suggestion?
I have a framework that relies on information sent by a custom Java Command 
Executor; think of some sort of heartbeat.  I start getting hearbeats after I 
send a task to that mesos-slave, but never before that.  That makes me assume 
that the CommandExecutor is not started until a task is submitted to be 
executed by that agent.  Is there a way to tell mesos-slave to start the 
ComandExecutor as soon as it starts running? Thanks,
Oscar
Thanks in advance
On Saturday, August 5, 2017, 6:08:37 AM EDT, Oeg Bizz  wrote:

I have a framework that relies on information sent by a custom Java Command 
Executor; think of some sort of heartbeat.  I start getting hearbeats after I 
send a task to that mesos-slave, but never before that.  That makes me assume 
that the CommandExecutor is not started until a task is submitted to be 
executed by that agent.  Is there a way to tell mesos-slave to start the 
ComandExecutor as soon as it starts running? Thanks,
Oscar