Re: Questions about Pods and the Mesos Containerizer

[Benjamin Mahler]

If moving the conversation to slack, it would be great to post back to the
list with a summary!

On Mon, Jan 29, 2018 at 1:38 PM, Vinod Kone  wrote:

> Hi David,
>
> It's probably worth having a synchronous discussion around your proposed
> approach in our slack. I would like to understand if TASK_GROUP is the
> right primitive for your use case.
>
> On Mon, Jan 29, 2018 at 1:32 PM, David Morrison  wrote:
>
>>
>>
>> On Thu, Jan 25, 2018 at 5:49 PM, Gilbert Song 
>> wrote:
>>
>>>
-

Is it possible to allocate a separate IP address per container in a
pod?

 Right now nested containers share the network from their parent
>>> container (pod). Do we have a specific use case that we need containers
>>> inside of a taskgroup have different IP addresses?
>>>
>>
>> For our use case, we need to be able to launch a relatively large number
>> of containers inside a taskgroup that all listen on the same port (and the
>> port is not easily-changeable).  So we need to be able to assign different
>> IPs to the containers so they don't conflict.
>>
>> Cheers,
>> David
>>
>
>

Re: Questions about Pods and the Mesos Containerizer

[Vinod Kone]

Hi David,

It's probably worth having a synchronous discussion around your proposed
approach in our slack. I would like to understand if TASK_GROUP is the
right primitive for your use case.

On Mon, Jan 29, 2018 at 1:32 PM, David Morrison  wrote:

>
>
> On Thu, Jan 25, 2018 at 5:49 PM, Gilbert Song 
> wrote:
>
>>
>>>-
>>>
>>>Is it possible to allocate a separate IP address per container in a
>>>pod?
>>>
>>> Right now nested containers share the network from their parent
>> container (pod). Do we have a specific use case that we need containers
>> inside of a taskgroup have different IP addresses?
>>
>
> For our use case, we need to be able to launch a relatively large number
> of containers inside a taskgroup that all listen on the same port (and the
> port is not easily-changeable).  So we need to be able to assign different
> IPs to the containers so they don't conflict.
>
> Cheers,
> David
>

Re: Questions about Pods and the Mesos Containerizer

[David Morrison]

On Thu, Jan 25, 2018 at 5:49 PM, Gilbert Song  wrote:

>
>>-
>>
>>Is it possible to allocate a separate IP address per container in a
>>pod?
>>
>> Right now nested containers share the network from their parent container
> (pod). Do we have a specific use case that we need containers inside of a
> taskgroup have different IP addresses?
>

For our use case, we need to be able to launch a relatively large number of
containers inside a taskgroup that all listen on the same port (and the
port is not easily-changeable).  So we need to be able to assign different
IPs to the containers so they don't conflict.

Cheers,
David

Re: Questions about Pods and the Mesos Containerizer

[Gilbert Song]

>
>
>-
>
>Is it possible to allocate a separate IP address per container in a
>pod?
>
> Right now nested containers share the network from their parent container
(pod). Do we have a specific use case that we need containers inside of a
taskgroup have different IP addresses?

- Gilbert

Re: Questions about Pods and the Mesos Containerizer

[Zhitao Li]

Glad someone is also looking this.

On Wed, Jan 24, 2018 at 2:43 PM, Jie Yu <yujie@gmail.com> wrote:

> I can help answer some of them:
>
> Is it possible to do healthchecks per task in a pod?
>
> I believe so given healthcheck is at the TaskInfo level, but AlexR can
> confirm.
>
>  Is it possible to allocate a separate IP address per container in a pod?
>
>  Not right now, but possible. We need to change the CNI network isolator
> to support that, but there might be caveats on the road.
>
> Is there any plan to support the Docker containeriser with pods?
>
> Probably not. If I want to do that, I'd prefer we refactor Docker
> containerizer to use containerd first, and then support pod there.
>
>  Timeframe for debugging tools (equivalent of docker exec, etc)?
>
> We'll have a containerization WG meeting tomorrow morning. I'll make sure
> this is on the list. Not timeframe yet, but this shouldn't take too long.
>
> Is there any performance data about using the Mesos containeriser with
>> container images versus using the Docker containeriser?
>> how does the Mesos containerizer handle extremely large images?
>>
>
We don't have systematic benchmark yet but we plan to do something in Q1,
after which we'll share some result. In an one-off case, we observed that
large image provisioning was ~2x faster on Mesos containerizer comparing to
docker daemon. The engineer believed it's because Linux tar was faster than
tar utilities from Golang's standard library. This is not repeatedly
verified.


> how does the Mesos containerizer handle dozens/hundreds of concurrent
>> pulls?
>
>
Same, we plan to do something and share our result once we have them.


>
> I believe Uber folks might have some data on this (cc Zhitao)?
>
> - Jie
>
> On Wed, Jan 24, 2018 at 2:21 PM, David Morrison <drm...@yelp.com> wrote:
>
>> Hi Mesos community!
>>
>> We’re in the process of designing a Mesos framework to launch multiple
>> containers together on the same host and are considering a couple of
>> approaches. The first is to use pods (with the TASK_GROUP primitive), and
>> the second is write a custom executor that launches nested containers and
>> use CNI to handle networking.
>>
>> With that in mind, we had the following questions:
>>
>> Questions about pods/task_groups:
>>
>>-
>>
>>    Is it possible to do healthchecks per task in a pod?
>>-
>>
>>Is it possible to allocate a separate IP address per container in a
>>pod?
>>-
>>
>>Is there any plan to support the Docker containeriser with pods?
>>
>>
>> Questions about UCR/Mesos containerizer:
>>
>>-
>>
>>Timeframe for debugging tools (equivalent of docker exec, etc)?
>>-
>>
>>Is there any performance data about using the Mesos containeriser
>>with container images versus using the Docker containeriser?
>>-
>>
>>   how does the Mesos containerizer handle extremely large images?
>>   -
>>
>>   how does the Mesos containerizer handle dozens/hundreds of
>>   concurrent pulls?
>>
>>
>> If anyone has had any experience using the UCR and/or pods with the sort
>> of workflow we’re considering, your input would be highly useful!
>>
>> Cheers,
>>
>> David Morrison
>>
>> Software Engineer @ Yelp
>>
>>
>


-- 
Cheers,

Zhitao Li

Re: Questions about Pods and the Mesos Containerizer

[Jie Yu]

I can help answer some of them:

Is it possible to do healthchecks per task in a pod?

I believe so given healthcheck is at the TaskInfo level, but AlexR can
confirm.

 Is it possible to allocate a separate IP address per container in a pod?

 Not right now, but possible. We need to change the CNI network isolator to
support that, but there might be caveats on the road.

Is there any plan to support the Docker containeriser with pods?

Probably not. If I want to do that, I'd prefer we refactor Docker
containerizer to use containerd first, and then support pod there.

 Timeframe for debugging tools (equivalent of docker exec, etc)?

We'll have a containerization WG meeting tomorrow morning. I'll make sure
this is on the list. Not timeframe yet, but this shouldn't take too long.

Is there any performance data about using the Mesos containeriser with
> container images versus using the Docker containeriser?
> how does the Mesos containerizer handle extremely large images?
> how does the Mesos containerizer handle dozens/hundreds of concurrent
> pulls?


I believe Uber folks might have some data on this (cc Zhitao)?

- Jie

On Wed, Jan 24, 2018 at 2:21 PM, David Morrison <drm...@yelp.com> wrote:

> Hi Mesos community!
>
> We’re in the process of designing a Mesos framework to launch multiple
> containers together on the same host and are considering a couple of
> approaches. The first is to use pods (with the TASK_GROUP primitive), and
> the second is write a custom executor that launches nested containers and
> use CNI to handle networking.
>
> With that in mind, we had the following questions:
>
> Questions about pods/task_groups:
>
>-
>
>Is it possible to do healthchecks per task in a pod?
>-
>
>Is it possible to allocate a separate IP address per container in a
>pod?
>-
>
>    Is there any plan to support the Docker containeriser with pods?
>
>
> Questions about UCR/Mesos containerizer:
>
>-
>
>Timeframe for debugging tools (equivalent of docker exec, etc)?
>-
>
>Is there any performance data about using the Mesos containeriser with
>container images versus using the Docker containeriser?
>-
>
>   how does the Mesos containerizer handle extremely large images?
>   -
>
>   how does the Mesos containerizer handle dozens/hundreds of
>   concurrent pulls?
>
>
> If anyone has had any experience using the UCR and/or pods with the sort
> of workflow we’re considering, your input would be highly useful!
>
> Cheers,
>
> David Morrison
>
> Software Engineer @ Yelp
>
>

Questions about Pods and the Mesos Containerizer

[David Morrison]

Hi Mesos community!

We’re in the process of designing a Mesos framework to launch multiple
containers together on the same host and are considering a couple of
approaches. The first is to use pods (with the TASK_GROUP primitive), and
the second is write a custom executor that launches nested containers and
use CNI to handle networking.

With that in mind, we had the following questions:

Questions about pods/task_groups:

   -

   Is it possible to do healthchecks per task in a pod?
   -

   Is it possible to allocate a separate IP address per container in a pod?
   -

   Is there any plan to support the Docker containeriser with pods?


Questions about UCR/Mesos containerizer:

   -

   Timeframe for debugging tools (equivalent of docker exec, etc)?
   -

   Is there any performance data about using the Mesos containeriser with
   container images versus using the Docker containeriser?
   -

  how does the Mesos containerizer handle extremely large images?
  -

  how does the Mesos containerizer handle dozens/hundreds of concurrent
  pulls?


If anyone has had any experience using the UCR and/or pods with the sort of
workflow we’re considering, your input would be highly useful!

Cheers,

David Morrison

Software Engineer @ Yelp