Re: Threshold-based CPU and Memory Oversubscription

[Vinod Kone]

Awesome. Great to see this!

Looking forward to the blog post on how this helped utilization in
production :P

On Wed, Sep 21, 2016 at 10:26 AM, Erb, Stephan 
wrote:

> Hi everyone,
>
>
>
> we are happy to announce that we have open sourced two simple
> threshold-based oversubscription modules for Mesos. We use them for CPU and
> memory oversubscription and have them running in production.
>
>
>
> https://github.com/blue-yonder/mesos-threshold-oversubscription
>
>
>
> The threshold-based approach enabled us to double the peak CPU and peak
> memory utilization in our Mesos/Aurora clusters. Your mileage may vary, so
> please take this statement with a grain of salt.
>
>
>
> Best Regards,
>
> Stephan Erb
>
> PS: Retweets welcome :-) https://twitter.com/BlueYonderTech/status/
> 778630174996893696
>

Re: multi-tenancy in mesos

[Rogier Dikkes]

Apache Aurora provides Multi tenancy and Quota's

http://aurora.apache.org/

http://aurora.apache.org/documentation/latest/features/multitenancy/

On 9/19/16 5:32 PM, haosdent wrote:

There is a topic about this in MesosCon EU.
http://schd.ws/hosted_files/mesosconeu2016/cd/compute_final_mesosConEur2016.pdf

On Mon, Sep 19, 2016 at 11:14 PM, tommy xiao > wrote:


Hi team,

anyone have some experience with multi-tenancy purpose build on
mesos cluster?
could you please share some hints? thanks a lot.

-- 
Deshi Xiao

Twitter: xds2000
E-mail: xiaods(AT)gmail.com 




--
Best Regards,
Haosdent Huang


--
Rogier Dikkes
Systeem Programmeur Hadoop & HPC Cloud
e-mail: rogier.dik...@surfsara.nl | M: +31 6 47 48 93 28
SURFsara | Science Park 140 | 1098 XG Amsterdam

Threshold-based CPU and Memory Oversubscription

[Erb, Stephan]

Hi everyone,

we are happy to announce that we have open sourced two simple threshold-based 
oversubscription modules for Mesos. We use them for CPU and memory 
oversubscription and have them running in production.

https://github.com/blue-yonder/mesos-threshold-oversubscription

The threshold-based approach enabled us to double the peak CPU and peak memory 
utilization in our Mesos/Aurora clusters. Your mileage may vary, so please take 
this statement with a grain of salt.

Best Regards,
Stephan Erb

PS: Retweets welcome :-) 
https://twitter.com/BlueYonderTech/status/778630174996893696

Re: Marathon API and support for pods

[John Sirois]

+1, good info - wrong forum.

On Wed, Sep 21, 2016 at 11:07 AM, Zameer Manji  wrote:

> Hey,
>
> Have you considered sending this to your framework's mailing list? As a
> Mesos user, I don't think framework specific documents like this need to be
> shared with the entire community.
>
>
> On Wed, Sep 21, 2016 at 2:59 AM, James DeFelice 
> wrote:
>
>> Hi folks,
>>
>> First of all the Marathon team would like to thank those who provided
>> feedback on the v3 API proposal (linked below) that was circulated last
>> month. Developing a new API for Marathon is a big undertaking and getting
>> your feedback early in the process has been helpful.
>>
>> "A vision for pods in Marathon
>> [https://docs.google.com/document/d/1uPH58NWN_OuynptsqTOq8v5
>> qlkivq2mUb2M9J5jZTMU/edit#heading=h.sqydeepp9s4m]
>>
>> We've done some additional discovery over the several weeks and have made
>> some changes to our API roadmap. It takes time to get an API right and the
>> decisions we make now will have lasting impact for months/years to come;
>> let's ensure that we spend enough time now thinking through the long-term
>> implications of particular API design choices. We'd also like to facilitate
>> a deeper discussion with the community about what problems v3 should solve
>> before committing to API decisions. The current plan is to resume work on
>> the v3 API later this fall. Stay tuned for additional announcements
>> regarding v3 API proposals.
>>
>> Furthermore, the v3 API is about more than just pods. We, the team and
>> community at large, should ensure that a new API is conceptually consistent
>> across API types and satisfies not only our short-term goals, but is
>> forward-compatible with our long term roadmap.
>>
>> That said, we have customer demand for pods now! In order to deliver pods
>> functionality without committing to a v3 API the team has decided to
>> introduce pods via a new /v2/pods API endpoint. What does this mean for v2
>> API users?
>>
>> First, if your organization doesn't have an interest in pods then nothing
>> forces you to change. Additional support for pods in the Marathon v2 API
>> should not cause breakage for existing v2 API users.
>>
>> Second, by integrating with the existing v2 API and backend we'll be
>> minimizing overall architectural changes. Needless to say there will be
>> changes to backend components that had been previously optimized for single
>> tasks vs. task groups. But the overall architecture of the system will not
>> change. This is important in order to preserve the performance,
>> scalability, and stability gains that Marathon has recently made.
>>
>> In addition, introducing pods in v2 allows the Marathon team to gather
>> early feedback from the community and our customers about how the API does
>> and does not meet their needs. This is very valuable input that will help
>> to shape the future v3 API.
>>
>> Below is a link to a proposal for pods in the Marathon v2 API. This
>> initial implementation for pods support should be viewed as an MVP that
>> will be enhanced in coming releases. Your feedback is most welcome and
>> strongly encouraged. Please comment directly in the document with any
>> questions or concerns.
>>
>> "Marathon: Pods in v2"
>> [https://docs.google.com/document/d/1Zno6QK2yGF4koB8BYT88EtB
>> 2-T7t3aAYRQ27pUD76gw/edit#heading=h.ywxj299mstr7]
>>
>> Many thanks,
>>
>> the Marathon team.
>>
>
>

Re: Marathon API and support for pods

[Zameer Manji]

Hey,

Have you considered sending this to your framework's mailing list? As a
Mesos user, I don't think framework specific documents like this need to be
shared with the entire community.


On Wed, Sep 21, 2016 at 2:59 AM, James DeFelice 
wrote:

> Hi folks,
>
> First of all the Marathon team would like to thank those who provided
> feedback on the v3 API proposal (linked below) that was circulated last
> month. Developing a new API for Marathon is a big undertaking and getting
> your feedback early in the process has been helpful.
>
> "A vision for pods in Marathon
> [https://docs.google.com/document/d/1uPH58NWN_
> OuynptsqTOq8v5qlkivq2mUb2M9J5jZTMU/edit#heading=h.sqydeepp9s4m]
>
> We've done some additional discovery over the several weeks and have made
> some changes to our API roadmap. It takes time to get an API right and the
> decisions we make now will have lasting impact for months/years to come;
> let's ensure that we spend enough time now thinking through the long-term
> implications of particular API design choices. We'd also like to facilitate
> a deeper discussion with the community about what problems v3 should solve
> before committing to API decisions. The current plan is to resume work on
> the v3 API later this fall. Stay tuned for additional announcements
> regarding v3 API proposals.
>
> Furthermore, the v3 API is about more than just pods. We, the team and
> community at large, should ensure that a new API is conceptually consistent
> across API types and satisfies not only our short-term goals, but is
> forward-compatible with our long term roadmap.
>
> That said, we have customer demand for pods now! In order to deliver pods
> functionality without committing to a v3 API the team has decided to
> introduce pods via a new /v2/pods API endpoint. What does this mean for v2
> API users?
>
> First, if your organization doesn't have an interest in pods then nothing
> forces you to change. Additional support for pods in the Marathon v2 API
> should not cause breakage for existing v2 API users.
>
> Second, by integrating with the existing v2 API and backend we'll be
> minimizing overall architectural changes. Needless to say there will be
> changes to backend components that had been previously optimized for single
> tasks vs. task groups. But the overall architecture of the system will not
> change. This is important in order to preserve the performance,
> scalability, and stability gains that Marathon has recently made.
>
> In addition, introducing pods in v2 allows the Marathon team to gather
> early feedback from the community and our customers about how the API does
> and does not meet their needs. This is very valuable input that will help
> to shape the future v3 API.
>
> Below is a link to a proposal for pods in the Marathon v2 API. This
> initial implementation for pods support should be viewed as an MVP that
> will be enhanced in coming releases. Your feedback is most welcome and
> strongly encouraged. Please comment directly in the document with any
> questions or concerns.
>
> "Marathon: Pods in v2"
> [https://docs.google.com/document/d/1Zno6QK2yGF4koB8BYT88EtB2-
> T7t3aAYRQ27pUD76gw/edit#heading=h.ywxj299mstr7]
>
> Many thanks,
>
> the Marathon team.
>

Re: what is the status on this?

[Alex Rukletsov]

Kant,

we would love to walk new community members through the code! We understand
how important it is to have a more experienced member of the community to
help out with patches, hence we have "shepherds". Moreover, though
technically possible, is not advised to start working without having
agreement with your shepherd.

Joseph Wu is driving the effort, get in touch with him and I'm sure you'll
figure out the plan!

On Tue, Sep 13, 2016 at 9:41 PM, kant kodali  wrote:

> @Alex Rukletsov I am sorry I took some time to respond. I am very excited
> since the beginning to have an opportunity to work on this task but I
> wanted to take my time if I can really commit to the Task and looks I might
> be able to however I have not contributed to open source before and I would
> need some help from someone who can point me to the right parts of the code
> and basically help me navigate through the process and if that is feasible
> I will be happy to commit some time every week to work on this. please let
> me know if that works.
>
>
>
> On Tue, Sep 6, 2016 11:59 AM, Dario Rexin dre...@apple.com wrote:
>
>> Frameworks would use the redirect mechanism of the HTTP API and in case
>> of unteachable nodes could do round robin on the list of master nodes.
>>
>> On Sep 6, 2016, at 11:52 AM, Joseph Wu  wrote:
>>
>> And for discovery of other nodes in the Paxos group.
>>
>> The work on modularizing/decoupling Zookeeper is a prerequisite for
>> having the replicated log perform leader election itself.  <- That would
>> merely be another implementation of the interface we will introduce in the
>> process:
>>
>> https://issues.apache.org/jira/browse/MESOS-3574
>>
>> On Tue, Sep 6, 2016 at 11:31 AM, Avinash Sridharan > > wrote:
>>
>> Also, I think, the replicated log itself uses Zookeeper for leader
>> election.
>>
>> On Tue, Sep 6, 2016 at 12:15 PM, Zameer Manji  wrote:
>>
>> If we use the replicated log for leader election, how will frameworks
>> detect the leading master? Right now the scheduler driver uses the
>> MasterInfo in ZK to discover the leader and detect leadership changes.
>>
>> On Mon, Sep 5, 2016 at 10:18 AM, Dario Rexin  wrote:
>>
>> If we go and change this, why not simply remove any dependencies to
>> external systems and simply use the replicated log for leader election?
>>
>> On Sep 5, 2016, at 9:02 AM, Alex Rukletsov  wrote:
>>
>> Kant—
>>
>> thanks a lot for the feedback! Are you interested in helping out with
>> Consul module once Jay and Joseph are done with modularizing patches?
>>
>> On Mon, Sep 5, 2016 at 8:50 AM, Jay JN Guo  wrote:
>>
>> Patches are currently under review by @Joseph and can be found at the
>> links provided by @haosdent.
>>
>> I took a quick look at Consul key/value HTTP APIs and they look very
>> similar to Etcd APIs. You could actually reuse our Etcd module
>> implementation once we manage to push the module into Mesos community.
>>
>> The only technical problem I could see for now is that Consul does not
>> support `POST` with incremental key index. We may need to leverage
>> `?cas=` operation in Consul to emulate the behaviour of joining a
>> key group.
>>
>> We could have a discussion on how to implement Consul HA module.
>>
>> cheers,
>> /J
>>
>>
>> - Original message -
>> From: haosdent 
>> To: user 
>> Cc: Jay JN Guo/China/IBM@IBMCN
>> Subject: Re: what is the status on this?
>> Date: Sun, Sep 4, 2016 6:10 PM
>>
>> Jay has some patches for de-couple Mesos with Zookeeper
>>
>> https://issues.apache.org/jira/browse/MESOS-5828
>> https://issues.apache.org/jira/browse/MESOS-5829
>>
>> I think it should be possible to support consul by custom modules after
>> jay's work done.
>>
>> On Sun, Sep 4, 2016 at 6:02 PM, kant kodali  wrote:
>>
>> Hi Alex,
>>
>> We have some experienced devops people here and they all had one thing in
>> common which is Zookeeper is a pain to maintain. In fact we refused to
>> bring in new tech stacks that require Zookeeper such as Kafka for example.
>> so we desperately in search for alternative preferably using consul. I just
>> hear lot of positive response when comes it consul. It will be great to see
>> mesos and consul working together in which we would be ready to jump at it
>> and make a switch for YARN to Mesos.
>>
>> Thanks,
>> Kant
>>
>>
>>
>>
>> On Wed, Aug 31, 2016 1:03 AM, Alex Rukletsov a...@mesosphere.com wrote:
>>
>> Kant—
>>
>> mind telling us what is your use case and why this ticket is important
>> for you? It will help us prioritize work.
>>
>> On Fri, Aug 26, 2016 at 2:46 AM, tommy xiao  wrote:
>>
>> Hi guys, i always focus on t his case. but good news is etcd always have
>> patchs. so the coming consul is very easy, just need some time to do coding
>> on it. if you have interesting it? let us collaborate it.
>>
>> 2016-08-26 8:11 GMT+08:00 Joseph Wu :
>>
>> There is no timeline as no one has done any work on the issue.
>>
>>
>> On Thu, Aug 25, 2016 at 4:54 PM, kant kodali  wrote:
>>
>> Hi Guys,
>>
>> I see this ticket and ot

Marathon API and support for pods

[James DeFelice]

Hi folks,

First of all the Marathon team would like to thank those who provided
feedback on the v3 API proposal (linked below) that was circulated last
month. Developing a new API for Marathon is a big undertaking and getting
your feedback early in the process has been helpful.

"A vision for pods in Marathon
[
https://docs.google.com/document/d/1uPH58NWN_OuynptsqTOq8v5qlkivq2mUb2M9J5jZTMU/edit#heading=h.sqydeepp9s4m
]

We've done some additional discovery over the several weeks and have made
some changes to our API roadmap. It takes time to get an API right and the
decisions we make now will have lasting impact for months/years to come;
let's ensure that we spend enough time now thinking through the long-term
implications of particular API design choices. We'd also like to facilitate
a deeper discussion with the community about what problems v3 should solve
before committing to API decisions. The current plan is to resume work on
the v3 API later this fall. Stay tuned for additional announcements
regarding v3 API proposals.

Furthermore, the v3 API is about more than just pods. We, the team and
community at large, should ensure that a new API is conceptually consistent
across API types and satisfies not only our short-term goals, but is
forward-compatible with our long term roadmap.

That said, we have customer demand for pods now! In order to deliver pods
functionality without committing to a v3 API the team has decided to
introduce pods via a new /v2/pods API endpoint. What does this mean for v2
API users?

First, if your organization doesn't have an interest in pods then nothing
forces you to change. Additional support for pods in the Marathon v2 API
should not cause breakage for existing v2 API users.

Second, by integrating with the existing v2 API and backend we'll be
minimizing overall architectural changes. Needless to say there will be
changes to backend components that had been previously optimized for single
tasks vs. task groups. But the overall architecture of the system will not
change. This is important in order to preserve the performance,
scalability, and stability gains that Marathon has recently made.

In addition, introducing pods in v2 allows the Marathon team to gather
early feedback from the community and our customers about how the API does
and does not meet their needs. This is very valuable input that will help
to shape the future v3 API.

Below is a link to a proposal for pods in the Marathon v2 API. This initial
implementation for pods support should be viewed as an MVP that will be
enhanced in coming releases. Your feedback is most welcome and strongly
encouraged. Please comment directly in the document with any questions or
concerns.

"Marathon: Pods in v2"
[
https://docs.google.com/document/d/1Zno6QK2yGF4koB8BYT88EtB2-T7t3aAYRQ27pUD76gw/edit#heading=h.ywxj299mstr7
]

Many thanks,

the Marathon team.

RE: Support for tasks groups aka pods in Mesos

[Hubert.Asamer]

Thanks for the clarification – I’ll try to dig into this and use execute.cpp  
as a starting point for my problem…
Kind Regards,
Hubert

From: Guangya Liu [mailto:gyliu...@gmail.com]
Sent: Mittwoch, 21. September 2016 09:19
To: user@mesos.apache.org
Cc: dev; Vinod Kone
Subject: Re: Support for tasks groups aka pods in Mesos

The answer is No, the taskGroup can be treated as Pod in Kubernetes, it will be 
a set of containers co-located and co-managed on an agent that share some 
resources (e.g., network namespace, volumes). For your case, you may want to 
split your 50 tasks to small task groups.

Also the `execute` cli is just an example framework and it can only launch one 
task group for now, you may want to enhance it if you want to launch multiple 
task groups.

On Wed, Sep 21, 2016 at 1:41 PM, 
mailto:hubert.asa...@dlr.de>> wrote:
Hi,
Very cool feature.
I’ve seen some recent changes 
(MESOS-6096) in the 
mesos-execute cli supporting grouped tasks which spawns the task-group on an 
arbitrary or specified agent node. Tested this yesterday and it works great.  
But if the task group holds a lot of one-off tasks (let’s say 50 tasks each 4 
cores,…) how could one achieve that  tasks are evenly distributed over several 
agents, depending on free resources? Don’t know if this is referred to pods, 
but do you think mesos-execute will support this?

Currently I’m using chronos & cook for batch scheduling, but I’d  like to 
achieve that every batch/group of  tasks runs in its own temporary framework 
like it does with mesos-execute

Thanks in advance & kind regards,
Hubert

——
Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR)
German Aerospace Center


German Remote Sensing Data Center | International Ground Segment | 
Oberpfaffenhofen | 82234 Wessling | Germany

Hubert Asamer
phone  +49 (0) 8153 28-2894 | fax +49 (0) 
8153 28-1443 | 
hubert.asa...@dlr.de
www.DLR.de/eoc




From: Vinod Kone [mailto:vinodk...@apache.org]
Sent: Dienstag, 9. August 2016 02:54
To: dev; user
Subject: Support for tasks groups aka pods in Mesos

Hi folks,

One of the most requested features in Mesos has been first class support for 
managing pod like containers. We finally have some time to focus and shepherd 
this work.

The epic tracking this work is : 
https://issues.apache.org/jira/browse/MESOS-2449

Design doc: https://issues.apache.org/jira/browse/MESOS-2449

Your feedback on the design will be most welcome. Once we get agreement on the 
design, we can start breaking down the epic into tickets.

Thanks,
Vinod & Jie

Re: Support for tasks groups aka pods in Mesos

[Guangya Liu]

The answer is No, the taskGroup can be treated as Pod in Kubernetes, it
will be a set of containers co-located and co-managed on an agent that
share some resources (e.g., network namespace, volumes). For your case, you
may want to split your 50 tasks to small task groups.

Also the `execute` cli is just an example framework and it can only launch
one task group for now, you may want to enhance it if you want to launch
multiple task groups.

On Wed, Sep 21, 2016 at 1:41 PM,  wrote:

> Hi,
>
> Very cool feature.
>
> I’ve seen some recent changes (MESOS-6096
> ) in the mesos-execute
> cli supporting grouped tasks which spawns the task-group on an arbitrary or
> specified agent node. Tested this yesterday and it works great.  But if the
> task group holds a lot of one-off tasks (let’s say 50 tasks each 4 cores,…)
> how could one achieve that  tasks are evenly distributed over several
> agents, depending on free resources? Don’t know if this is referred to
> pods, but do you think mesos-execute will support this?
>
>
>
> Currently I’m using chronos & cook for batch scheduling, but I’d  like to
> achieve that every batch/group of  tasks runs in its own temporary
> framework like it does with mesos-execute
>
>
>
> Thanks in advance & kind regards,
>
> Hubert
>
>
>
> ——
>
> *Deutsches Zentrum für Luft- und Raumfahrt* e.V. (DLR)
>
> German Aerospace Center
>
>
>
>
>
> German Remote Sensing Data Center | International Ground Segment |
> Oberpfaffenhofen | 82234 Wessling | Germany
>
>
>
> *Hubert Asamer*
>
> phone  +49 (0) 8153 28-2894 | fax +49 (0) 8153 28-1443 |
> hubert.asa...@dlr.de
>
> www.DLR.de/eoc 
>
>
>
>
>
>
>
>
>
> *From:* Vinod Kone [mailto:vinodk...@apache.org]
> *Sent:* Dienstag, 9. August 2016 02:54
> *To:* dev; user
> *Subject:* Support for tasks groups aka pods in Mesos
>
>
>
> Hi folks,
>
>
>
> One of the most requested features in Mesos has been first class support
> for managing pod like containers. We finally have some time to focus and
> shepherd this work.
>
>
>
> The epic tracking this work is : https://issues.apache.org/
> jira/browse/MESOS-2449
>
>
>
> Design doc: https://issues.apache.org/jira/browse/MESOS-2449
>
>
>
> Your feedback on the design will be most welcome. Once we get agreement on
> the design, we can start breaking down the epic into tickets.
>
>
>
> Thanks,
>
> Vinod & Jie
>