Re: GenOuest makes use of Mesos

[Elouan Keryell-Even]

Thanks for the insight, very interesting indeed.

Elouan Keryell-Even

Software Engineer @ Atos Integration

Toulouse, France

2015-12-09 11:08 GMT+01:00 Olivier Sallou :

>
>
> --
>
> *De: *"Elouan Keryell-Even" 
> *À: *user@mesos.apache.org
> *Envoyé: *Mercredi 9 Décembre 2015 10:48:10
> *Objet: *Re: GenOuest makes use of Mesos
>
> Hi Olivier,
>
> I've just read the presentation on your project's webpage, and it seems
> cool! I'm curious about the following mentioned feature: "Optional mount of
> user home or other shared directories in container". Does your framework
> take care of remotely copying the home directory onto the node where the
> container is going to run, or does the directory have to be already
> available (as a shared directory for example).
>
> We focus to "replace" classical computing cluster frameworks like Sun Grid
> Engine. So home directories etc... are shared among nodes (nfs, etc...),
> like in a usual cluster environment.
> The mount of "user home dir" is based on an Auth/ACL plugin/configuration.
> Admin specifies the available mounts for all users, or on a project basis.
> Then the Auth/ACL plugin maps those volumes to real available directories
> (if using the LDAP auth plugin, then get homeDirectory from LDAP and add as
> a volume to container).
>
> Otherwise, we are currently using framework Chronos for our Batch oriented
> containers, and your framework seems to fit the same spot. If you have some
> experience with Chronos, I'd be interested in a brief comparison of both
> frameworks. From what I understand, GODocker offers scheduling policy's
> customization, which I don't think Chronos does.
>
> There is indeed scheduling algorithms plugin mechanism. An interesting one
> is the fair-share policy imlpementation, where tasks are scheduler
> (ordered) based on previous user/project consumption.
>
> Difference with Chronos, is Chronos is made for cron like jobs. GODocker
> is a batch submission tool. User specifies the computing script he wants to
> execute and it is executed "immediatly". The scheduling is a matter of
> putting priorities when executing jobs (if users submits 1000 jobs but I
> have only 100 slots available). We focus on users submitting "many" jobs or
> many users submitting jobs, each job could be a computing task of a few
> seconds or several days.
>
>
>
> Olivier
>
>
> Thanks for your attention,
>
> Elouan Keryell-Even
>
> Software Engineer @ Atos Integration
>
> Toulouse, France
>
> 2015-12-08 14:49 GMT+01:00 Olivier Sallou :
>
>> Hi,
>> the GenOuest (http://www.genouest.org) academic lab is now using Mesos
>> in production in its core facility to manage scientists computing tasks
>> (for bioinformatics)
>> To do so, we have developed a new mesos framework, GoDocker
>> (http://www.genouest.org/godocker) to submit batch computing scripts on
>> premises. It mounts users home directory or other shared resources to
>> execute jobs in Docker containers, using Mesos as main scheduler.
>> GoDocker schedules the jobs according to user/groups priorities and
>> quotas and provides a CLI, a REST web interface and a partial DRMAA
>> library support. Framewok is open source.
>>
>> Thanks for adding us to the Mesos fellows ;-)
>>
>> Regards
>>
>> Olivier (GenOuest core developer member)
>>
>> --
>> Olivier Sallou
>> IRISA / University of Rennes 1
>> Campus de Beaulieu, 35000 RENNES - FRANCE
>> Tel: 02.99.84.71.95
>>
>> gpg key id: 4096R/326D8438  (keyring.debian.org)
>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438
>>
>>
>>
>
>

Re: GenOuest makes use of Mesos

[Olivier Sallou]

- Mail original -

> De: "Elouan Keryell-Even" 
> À: user@mesos.apache.org
> Envoyé: Mercredi 9 Décembre 2015 10:48:10
> Objet: Re: GenOuest makes use of Mesos

> Hi Olivier,

> I've just read the presentation on your project's webpage, and it seems cool!
> I'm curious about the following mentioned feature: "Optional mount of user
> home or other shared directories in container". Does your framework take
> care of remotely copying the home directory onto the node where the
> container is going to run, or does the directory have to be already
> available (as a shared directory for example).

We focus to "replace" classical computing cluster frameworks like Sun Grid 
Engine. So home directories etc... are shared among nodes (nfs, etc...), like 
in a usual cluster environment. 
The mount of "user home dir" is based on an Auth/ACL plugin/configuration. 
Admin specifies the available mounts for all users, or on a project basis. Then 
the Auth/ACL plugin maps those volumes to real available directories (if using 
the LDAP auth plugin, then get homeDirectory from LDAP and add as a volume to 
container). 

> Otherwise, we are currently using framework Chronos for our Batch oriented
> containers, and your framework seems to fit the same spot. If you have some
> experience with Chronos, I'd be interested in a brief comparison of both
> frameworks. From what I understand, GODocker offers scheduling policy's
> customization, which I don't think Chronos does.

There is indeed scheduling algorithms plugin mechanism. An interesting one is 
the fair-share policy imlpementation, where tasks are scheduler (ordered) based 
on previous user/project consumption. 

Difference with Chronos, is Chronos is made for cron like jobs. GODocker is a 
batch submission tool. User specifies the computing script he wants to execute 
and it is executed "immediatly". The scheduling is a matter of putting 
priorities when executing jobs (if users submits 1000 jobs but I have only 100 
slots available). We focus on users submitting "many" jobs or many users 
submitting jobs, each job could be a computing task of a few seconds or several 
days. 

Olivier 

> Thanks for your attention,

> Elouan Keryell-Even

> Software Engineer @ Atos Integration

> Toulouse, France

> 2015-12-08 14:49 GMT+01:00 Olivier Sallou < olivier.sal...@irisa.fr > :

> > Hi,
> 
> > the GenOuest ( http://www.genouest.org ) academic lab is now using Mesos
> 
> > in production in its core facility to manage scientists computing tasks
> 
> > (for bioinformatics)
> 
> > To do so, we have developed a new mesos framework, GoDocker
> 
> > ( http://www.genouest.org/godocker ) to submit batch computing scripts on
> 
> > premises. It mounts users home directory or other shared resources to
> 
> > execute jobs in Docker containers, using Mesos as main scheduler.
> 
> > GoDocker schedules the jobs according to user/groups priorities and
> 
> > quotas and provides a CLI, a REST web interface and a partial DRMAA
> 
> > library support. Framewok is open source.
> 

> > Thanks for adding us to the Mesos fellows ;-)
> 

> > Regards
> 

> > Olivier (GenOuest core developer member)
> 

> > --
> 
> > Olivier Sallou
> 
> > IRISA / University of Rennes 1
> 
> > Campus de Beaulieu, 35000 RENNES - FRANCE
> 
> > Tel: 02.99.84.71.95
> 

> > gpg key id: 4096R/326D8438 ( keyring.debian.org )
> 
> > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
> 

Re: GenOuest makes use of Mesos

[Elouan Keryell-Even]

Hi Olivier,

I've just read the presentation on your project's webpage, and it seems
cool! I'm curious about the following mentioned feature: "Optional mount of
user home or other shared directories in container". Does your framework
take care of remotely copying the home directory onto the node where the
container is going to run, or does the directory have to be already
available (as a shared directory for example).

Otherwise, we are currently using framework Chronos for our Batch oriented
containers, and your framework seems to fit the same spot. If you have some
experience with Chronos, I'd be interested in a brief comparison of both
frameworks. From what I understand, GODocker offers scheduling policy's
customization, which I don't think Chronos does.

Thanks for your attention,

Elouan Keryell-Even

Software Engineer @ Atos Integration

Toulouse, France

2015-12-08 14:49 GMT+01:00 Olivier Sallou :

> Hi,
> the GenOuest (http://www.genouest.org) academic lab is now using Mesos
> in production in its core facility to manage scientists computing tasks
> (for bioinformatics)
> To do so, we have developed a new mesos framework, GoDocker
> (http://www.genouest.org/godocker) to submit batch computing scripts on
> premises. It mounts users home directory or other shared resources to
> execute jobs in Docker containers, using Mesos as main scheduler.
> GoDocker schedules the jobs according to user/groups priorities and
> quotas and provides a CLI, a REST web interface and a partial DRMAA
> library support. Framewok is open source.
>
> Thanks for adding us to the Mesos fellows ;-)
>
> Regards
>
> Olivier (GenOuest core developer member)
>
> --
> Olivier Sallou
> IRISA / University of Rennes 1
> Campus de Beaulieu, 35000 RENNES - FRANCE
> Tel: 02.99.84.71.95
>
> gpg key id: 4096R/326D8438  (keyring.debian.org)
> Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438
>
>
>

Re: GenOuest makes use of Mesos

[Olivier Sallou]

- Mail original -

> De: "Benjamin Mahler" 
> À: user@mesos.apache.org
> Envoyé: Mardi 8 Décembre 2015 22:33:40
> Objet: Re: GenOuest makes use of Mesos

> Great to hear Olivier, would you like to be added to the powered by mesos
> list?

> https://github.com/apache/mesos/blob/master/docs/powered-by-mesos.md

Sure 

Should the framework be also added to 
https://github.com/apache/mesos/blob/master/docs/frameworks.md ? 

Olivier 

> On Tue, Dec 8, 2015 at 1:04 PM, Arunabha Ghosh < arunabha...@gmail.com >
> wrote:

> > Welcome to the community, Oliver.
> 

> > On Tue, Dec 8, 2015 at 5:49 AM, Olivier Sallou < olivier.sal...@irisa.fr >
> > wrote:
> 

> > > Hi,
> > 
> 
> > > the GenOuest ( http://www.genouest.org ) academic lab is now using Mesos
> > 
> 
> > > in production in its core facility to manage scientists computing tasks
> > 
> 
> > > (for bioinformatics)
> > 
> 
> > > To do so, we have developed a new mesos framework, GoDocker
> > 
> 
> > > ( http://www.genouest.org/godocker ) to submit batch computing scripts on
> > 
> 
> > > premises. It mounts users home directory or other shared resources to
> > 
> 
> > > execute jobs in Docker containers, using Mesos as main scheduler.
> > 
> 
> > > GoDocker schedules the jobs according to user/groups priorities and
> > 
> 
> > > quotas and provides a CLI, a REST web interface and a partial DRMAA
> > 
> 
> > > library support. Framewok is open source.
> > 
> 

> > > Thanks for adding us to the Mesos fellows ;-)
> > 
> 

> > > Regards
> > 
> 

> > > Olivier (GenOuest core developer member)
> > 
> 

> > > --
> > 
> 
> > > Olivier Sallou
> > 
> 
> > > IRISA / University of Rennes 1
> > 
> 
> > > Campus de Beaulieu, 35000 RENNES - FRANCE
> > 
> 
> > > Tel: 02.99.84.71.95
> > 
> 

> > > gpg key id: 4096R/326D8438 ( keyring.debian.org )
> > 
> 
> > > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
> > 
> 

Re: GenOuest makes use of Mesos

[Joao Ribeiro]

Hi Oliver,

GoDocker looks very promising. I have been looking for a a framework to 
substitute AWS Lambda for batch jobs scheduling based on docker for 
flexibility. That said, I would like to know your thought on this to better 
understand if GoDocker could do the job.
A couple of things I would need to know better:

- How fast can GoDocker launch a job? Can GoDocker pre populate all nodes with 
the container and files needed for the job to achieve <500ms launches? I 
realize this depends a lot on the Job size, just looking for a generic view 
before getting into full testing.
- Is it possible to extend GoDocker to enable usage metrics reports? This could 
be achieved by simply logging a few metrics relating to the user and job.


{ name: “João Ribeiro”,
  email: “jonnyb...@gmail.com ”,
  phone: “+351 916 002 675 ”,
  skype: “jmn_ribeiro ”,
  linkedin: “https://pt.linkedin.com/in/jmnribeiro 
”
  github: “https://github.com/JonnyBGod ” }

> On 09 Dec 2015, at 10:22, Elouan Keryell-Even  
> wrote:
> 
> Thanks for the insight, very interesting indeed.
> 
> Elouan Keryell-Even
> Software Engineer @ Atos Integration
> 
> Toulouse, France
> 
> 
> 2015-12-09 11:08 GMT+01:00 Olivier Sallou  >:
> 
> 
> De: "Elouan Keryell-Even"  >
> À: user@mesos.apache.org 
> Envoyé: Mercredi 9 Décembre 2015 10:48:10
> Objet: Re: GenOuest makes use of Mesos
> 
> Hi Olivier,
> 
> I've just read the presentation on your project's webpage, and it seems cool! 
> I'm curious about the following mentioned feature: "Optional mount of user 
> home or other shared directories in container". Does your framework take care 
> of remotely copying the home directory onto the node where the container is 
> going to run, or does the directory have to be already available (as a shared 
> directory for example).
> We focus to "replace" classical computing cluster frameworks like Sun Grid 
> Engine. So home directories etc... are shared among nodes (nfs, etc...), like 
> in a usual cluster environment.
> The mount of "user home dir" is based on an Auth/ACL plugin/configuration. 
> Admin specifies the available mounts for all users, or on a project basis. 
> Then the Auth/ACL plugin maps those volumes to real available directories (if 
> using the LDAP auth plugin, then get homeDirectory from LDAP and add as a 
> volume to container).
> Otherwise, we are currently using framework Chronos for our Batch oriented 
> containers, and your framework seems to fit the same spot. If you have some 
> experience with Chronos, I'd be interested in a brief comparison of both 
> frameworks. From what I understand, GODocker offers scheduling policy's 
> customization, which I don't think Chronos does.
> There is indeed scheduling algorithms plugin mechanism. An interesting one is 
> the fair-share policy imlpementation, where tasks are scheduler (ordered) 
> based on previous user/project consumption.
> 
> Difference with Chronos, is Chronos is made for cron like jobs. GODocker is a 
> batch submission tool. User specifies the computing script he wants to 
> execute and it is executed "immediatly". The scheduling is a matter of 
> putting priorities when executing jobs (if users submits 1000 jobs but I have 
> only 100 slots available). We focus on users submitting "many" jobs or many 
> users submitting jobs, each job could be a computing task of a few seconds or 
> several days.
> 
> 
> 
> Olivier
> 
> 
> Thanks for your attention,
> 
> Elouan Keryell-Even
> Software Engineer @ Atos Integration
> 
> Toulouse, France
> 
> 
> 2015-12-08 14:49 GMT+01:00 Olivier Sallou  >:
> Hi,
> the GenOuest (http://www.genouest.org ) academic 
> lab is now using Mesos
> in production in its core facility to manage scientists computing tasks
> (for bioinformatics)
> To do so, we have developed a new mesos framework, GoDocker
> (http://www.genouest.org/godocker ) to 
> submit batch computing scripts on
> premises. It mounts users home directory or other shared resources to
> execute jobs in Docker containers, using Mesos as main scheduler.
> GoDocker schedules the jobs according to user/groups priorities and
> quotas and provides a CLI, a REST web interface and a partial DRMAA
> library support. Framewok is open source.
> 
> Thanks for adding us to the Mesos fellows ;-)
> 
> Regards
> 
> Olivier (GenOuest core developer member)
> 
> --
> Olivier Sallou
> IRISA / University of Rennes 1
> Campus de Beaulieu, 35000 RENNES - FRANCE
> Tel: 02.99.84.71.95 
> 
> gpg key id: 4096R/326D8438  (keyring.debian.org )
> Key fingerprint = 

Upgrade 0.22.1 --> 0.25.0

[aaaler]

Hi All!

We have running cluster with mesos 0.22.1 and marathon 0.8.2 running on it.
Is it safe to upgrade directly 0.22.1 --> 0.25.0 without installing
intermediate versions?

How i can perform rollback if something went wrong? Would installing
previous versions just do the job?

--
 Alex Griazin

Re: Upgrade 0.22.1 --> 0.25.0

[tommy xiao]

Hi Alex,

I think you need review the mesos upgrade docs.

http://mesos.apache.org/documentation/latest/upgrades/



2015-12-09 23:04 GMT+08:00 :

> Hi All!
>
> We have running cluster with mesos 0.22.1 and marathon 0.8.2 running on it.
> Is it safe to upgrade directly 0.22.1 --> 0.25.0 without installing
> intermediate versions?
>
> How i can perform rollback if something went wrong? Would installing
> previous versions just do the job?
>
> --
>  Alex Griazin
>



-- 
Deshi Xiao
Twitter: xds2000
E-mail: xiaods(AT)gmail.com

Re: Upgrade 0.22.1 --> 0.25.0

[Brandon Gulla]

Everything I have heard in the past says to stick with incremental
upgrades. I actually performed the same upgrade as you just last week.
Stick with the 0.x.1 releases and you should be fine. The only trouble I
ran into was at some point my Task Counters went missing from the web-ui
but reappeared after about 20 minutes.

Consult the Docs for changes between the versions.
http://mesos.apache.org/documentation/latest/upgrades/


Good luck.

On Wed, Dec 9, 2015 at 10:04 AM,  wrote:

> Hi All!
>
> We have running cluster with mesos 0.22.1 and marathon 0.8.2 running on it.
> Is it safe to upgrade directly 0.22.1 --> 0.25.0 without installing
> intermediate versions?
>
> How i can perform rollback if something went wrong? Would installing
> previous versions just do the job?
>
> --
>  Alex Griazin
>



-- 
Brandon

Re: Sync Mesos-Master to Slaves

[Alex Rukletsov]

Frederic,

I have skimmed through the logs and they are do not seem to be complete
(especially for master1). Could you please say what task has been killed
(id) and which master failover triggered that? I see at least three
failovers in the logs : ). Also, could you please share some background
about your setup? I believe you're on systemd, do you use docker tasks?

To connect our conversation to particular events, let me post here the
chain of (potentially) interesting events and some info I mined from the
logs.
master1: 192.168.37.59 ?
master2: 192.168.37.58
master3: 192.168.37.104

timestamp   observed by   event
13:48:38 master1  master1 killed by sigterm
13:48:48 master2,3   new leader elected (192.168.37.104), id=5
13:49:25 master2  master2 killed by sigterm
13:50:44 master2,3   new leader elected (192.168.37.59), id=7
14:23:34 master1  master1 killed by sigterm
14:23:44 master2,3   new leader elected (192.168.37.58), id=8

One interesting thing I cannot understand is why master3 did not commit
suicide when it lost leadership?


On Mon, Dec 7, 2015 at 4:08 PM, Frederic LE BRIS 
wrote:

> With the context .. sorry
>
>

Re: [VOTE] Release Apache Mesos 0.26.0 (rc4)

[Benjamin Mahler]

I'd really like to pull in the fix for:
https://issues.apache.org/jira/browse/MESOS-4106

This has been a long standing bug that makes the health checking not
function correctly some of the time. While it is rare in CI, it appeared in
a colleague's cluster for about a third of the tasks he was launching to
demonstrate how he ran into this. The fix is trivial and is in review.

Thoughts?

On Tue, Dec 8, 2015 at 7:01 AM, Bernd Mathiske  wrote:

> +1 (binding)
>
> Ran make check, make distcheck, sudo bin/mesos-tests.sh, with SSL enabled
> and without on: Ubuntu 12.04, CentOS 7.1.
>
> Had 4 test failures with CentOS 7.1 for each configuration variant. All of
> the failed tests are known to be flaky, they have MESOS tickets, and they
> have been investigated and are deemed non-blockers.
>
> Bernd
>
> > On Dec 8, 2015, at 4:59 AM, Till Toenshoff  wrote:
> >
> > Hi friends,
> >
> > we had noticed some discrepancies between the V0 API and the V1 API,
> > hence we had to create a new release candidate even after the voting of
> > 0.26.0-rc3 had officially ended. Sorry for that!
> >
> > Please vote on releasing the following candidate as Apache Mesos 0.26.0.
> >
> > The CHANGELOG for the release is available at:
> >
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.26.0-rc4
> >
> 
> >
> > The candidate for Mesos 0.26.0 release is available at:
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc4/mesos-0.26.0.tar.gz
> >
> > The tag to be voted on is 0.26.0-rc4:
> >
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.26.0-rc4
> >
> > The MD5 checksum of the tarball can be found at:
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc4/mesos-0.26.0.tar.gz.md5
> >
> > The signature of the tarball can be found at:
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc4/mesos-0.26.0.tar.gz.asc
> >
> > The PGP key used to sign the release is here:
> > https://dist.apache.org/repos/dist/release/mesos/KEYS
> >
> > The JAR is up in Maven in a staging repository here:
> > https://repository.apache.org/content/repositories/orgapachemesos-1093
> >
> > Please vote on releasing this package as Apache Mesos 0.26.0!
> >
> > The vote is open until Fri Dec 11 04:50:51 CET 2015 and passes if a
> majority of at least 3 +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Mesos 0.26.0
> > [ ] -1 Do not release this package because ...
> >
> > Thanks,
> > Bernd & Till
> >
>
>

Re: Mesos ACL User

[Vinod Kone]

Mesos doesn't understand user groups yet, so you can't setup ACLs on them.

On Wed, Dec 9, 2015 at 11:33 AM, John Omernik  wrote:

> Well the mesos slaves are running as root, so they can run any task as any
> user. What I believe this is trying to do is using authentication in Mesos,
> ensure that only certain folks, with the secret can run as certain users.
> Thus, if I have a prn_devcontrol principal and secret, that can be used by
> a marathon framework I start. My devs can use that framework, and now they
> can only run tasks as themselves. Now, obviously there are concerns here...
> a single shared password defeats accountability, This would not be a
> concern if I gave a principal to each user.  Of course, then I have to
> restart mesos masters each time I change the ACLs which is also a
> challenge.
>
>
>
> On Wed, Dec 9, 2015 at 11:52 AM, haosdent  wrote:
>
>> >That it would allow a task to run in the devmarathon as any unix user
>> in that group.
>> I think in Linux only root would run task with other user. For normal
>> users, although they are in same group, it also not allowed to run task
>> with other account except use sudo. So "roles" in ACL is more close to what
>> you expect here?
>>
>> On Wed, Dec 9, 2015 at 12:12 AM, John Omernik  wrote:
>>
>>> In crafting my ACLs, I found that I would like to have a situation where
>>> groups were used instead of just user... i.e. if I have a certain frame,
>>> perhaps a dev instance of Marathon, I want folks in the dev group to all be
>>> able to to run frameworks as themselves.  Right now,  have a principal that
>>> can run in any role and with any user, prn_prodcontrol. That works for me.
>>> Then I have a principal that is my devcontrol.  So I register dev Marathon
>>> with that, and now anyone who has my credentials for the dev marathon, can
>>> submit marathon jobs, which is cool, however, they can only do it as
>>> unixdevuser, which is my unix user on every box... that's cool too. Also,
>>> the marathondev framework can only operate in the dev role.
>>>
>>>
>>> {
>>>  "register_frameworks": [
>>>   { "principals": { "values": ["prn_prodcontrol"] }, "roles": { "type":
>>> "ANY"}},
>>>   { "principals": { "values": ["prn_devcontrol"] }, "roles": {"values":
>>> ["dev"]}}
>>>   ]
>>>  "run_tasks": [
>>>   { "principals": { "values": ["prn_prodcontrol"] }, "users": { "type":
>>> "ANY"}},
>>>   { "principals": { "values": ["prn_devcontrol"] }, "users": {"values":
>>> ["unixdevuser"]}}
>>> ]
>>> }
>>>
>>>
>>> What would be ideal is if I have a group marathondevgrp (unix group on
>>> all nodes) and then I register the marathondev framework with principle
>>> prn_devcontrol, having an ACL that stated...
>>>
>>>
>>> {
>>>  "register_frameworks": [
>>>   { "principals": { "values": ["prn_prodcontrol"] }, "roles": { "type":
>>> "ANY"}},
>>>   { "principals": { "values": ["prn_devcontrol"] }, "roles": {"values":
>>> ["dev"]}}
>>>   ]
>>>  "run_tasks": [
>>>   { "principals": { "values": ["prn_prodcontrol"] }, "users": { "type":
>>> "ANY"}},
>>>   { "principals": { "values": ["prn_devcontrol"] }, "users": {"values":
>>> ["marathondevgrp"]}}
>>> ]
>>> }
>>>
>>> That it would allow a task to run in the devmarathon as any unix user in
>>> that group. This would allow me to have dev users run frameworks as
>>> themselves (for data access control on my shared filesystem) and still have
>>> the freedom to submit to marathon (dev).
>>>
>>>
>>> So does ACLs support groups? Is this something that would be difficult
>>> to add?  Thoughts about other approach to achieve similar results?
>>>
>>> Thanks!
>>>
>>> John
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>
>

Re: Mesos ACL User

[haosdent]

>That it would allow a task to run in the devmarathon as any unix user in
that group.
I think in Linux only root would run task with other user. For normal
users, although they are in same group, it also not allowed to run task
with other account except use sudo. So "roles" in ACL is more close to what
you expect here?

On Wed, Dec 9, 2015 at 12:12 AM, John Omernik  wrote:

> In crafting my ACLs, I found that I would like to have a situation where
> groups were used instead of just user... i.e. if I have a certain frame,
> perhaps a dev instance of Marathon, I want folks in the dev group to all be
> able to to run frameworks as themselves.  Right now,  have a principal that
> can run in any role and with any user, prn_prodcontrol. That works for me.
> Then I have a principal that is my devcontrol.  So I register dev Marathon
> with that, and now anyone who has my credentials for the dev marathon, can
> submit marathon jobs, which is cool, however, they can only do it as
> unixdevuser, which is my unix user on every box... that's cool too. Also,
> the marathondev framework can only operate in the dev role.
>
>
> {
>  "register_frameworks": [
>   { "principals": { "values": ["prn_prodcontrol"] }, "roles": { "type":
> "ANY"}},
>   { "principals": { "values": ["prn_devcontrol"] }, "roles": {"values":
> ["dev"]}}
>   ]
>  "run_tasks": [
>   { "principals": { "values": ["prn_prodcontrol"] }, "users": { "type":
> "ANY"}},
>   { "principals": { "values": ["prn_devcontrol"] }, "users": {"values":
> ["unixdevuser"]}}
> ]
> }
>
>
> What would be ideal is if I have a group marathondevgrp (unix group on all
> nodes) and then I register the marathondev framework with principle
> prn_devcontrol, having an ACL that stated...
>
>
> {
>  "register_frameworks": [
>   { "principals": { "values": ["prn_prodcontrol"] }, "roles": { "type":
> "ANY"}},
>   { "principals": { "values": ["prn_devcontrol"] }, "roles": {"values":
> ["dev"]}}
>   ]
>  "run_tasks": [
>   { "principals": { "values": ["prn_prodcontrol"] }, "users": { "type":
> "ANY"}},
>   { "principals": { "values": ["prn_devcontrol"] }, "users": {"values":
> ["marathondevgrp"]}}
> ]
> }
>
> That it would allow a task to run in the devmarathon as any unix user in
> that group. This would allow me to have dev users run frameworks as
> themselves (for data access control on my shared filesystem) and still have
> the freedom to submit to marathon (dev).
>
>
> So does ACLs support groups? Is this something that would be difficult to
> add?  Thoughts about other approach to achieve similar results?
>
> Thanks!
>
> John
>
>
>
>
>


-- 
Best Regards,
Haosdent Huang

Re: Upgrade 0.22.1 --> 0.25.0

[Adam Bordelon]

Please do not skip versions during upgrades. We have been using a
two-version deprecation cycle, so a protobuf field that existed in 0.22.1
may no longer exist in 0.24.x, meaning that a direct live upgrade could
cause masters and agents to stop communicating with each other. We only
guarantee live upgrades from one version to the next minor version (e.g.
0.22.x->0.23.x), so the recommended procedure is to follow the linked
upgrade guide to upgrade the entire cluster (even the libmesos for
schedulers/executors) to the next version, then repeat the procedure to
upgrade the entire cluster to the following version and so on.
The usual process is to upgrade masters first, then agents, then schedulers
and custom executors.
There may be other caveats listed in the upgrades guide that indicate
deprecated/removed endpoints, formatting/behavior changes, etc.

Now that we've moved to a more monthly release cycle, we will be extending
our deprecation cycle to ~6months/versions, so that you could skip versions
in the future. Once we reach 1.0, we'll have stronger guarantees about
upgrading minor versions within 1.x.

On Wed, Dec 9, 2015 at 7:39 AM, tommy xiao  wrote:

> Hi Alex,
>
> I think you need review the mesos upgrade docs.
>
> http://mesos.apache.org/documentation/latest/upgrades/
>
>
>
> 2015-12-09 23:04 GMT+08:00 :
>
>> Hi All!
>>
>> We have running cluster with mesos 0.22.1 and marathon 0.8.2 running on
>> it.
>> Is it safe to upgrade directly 0.22.1 --> 0.25.0 without installing
>> intermediate versions?
>>
>> How i can perform rollback if something went wrong? Would installing
>> previous versions just do the job?
>>
>> --
>>  Alex Griazin
>>
>
>
>
> --
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com
>