@Neil, just want to confirm about ACL, do you mean we will load role info from 3rd part application, e.g. LDAP?
And as I mentioned in both design doc, why not build a RoleManager as plugin for them? Both features are required following operator: 1. check: check whether role is available 2. create: create role in Master 3. update: update role info 4. destroy: delete the role 5. persist: 6. query: query from role manager. master/allocator need role info during the operation For the life cycle of role, "update" should be period & manual (API/HTT); others are trigger by API/HTTP. ---- Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer Platform Symphony/DCOS Development & Support, STG, IBM GCG +86-10-8245 4084 | [email protected] | http://k82.me On Tue, Dec 1, 2015 at 1:13 PM, James Peach <[email protected]> wrote: > > > On Nov 30, 2015, at 6:53 PM, YongQiao Wang <[email protected]> > wrote: > > > > Hi All, > > > > Currently, there are two proposals on how to improve role management in > > Mesos: > > > > (a) Dynamic roles (MESOS-3177): roles are stored in the registry > > and queried/added/deleted/removed via HTTP endpoints. I posted a design > doc > > here: > > > https://docs.google.com/document/d/1OIgceqpsjV3-_LGF83IMAFnrh1Ea3Zc16w9kWWPpUj4/edit# > > > > (b) Implicit Roles (MESOS-3988): any role will be allowed, subject to the > > ACL/authorization system. In a sense, "all roles" exist, so there is no > > need to store the set of legal roles or provide endpoints to modify them. > > Neil also has posted a design doc here: > > > https://docs.google.com/document/d/1SCFfrBd4edSY3bVCMrNJYMxIVllD0bHJuGmgG-4vCXA/edit?usp=sharing > > Implicit roles seem to behave more like groups in an OS where the role > string is more gid_t which can be consumed directly by the kernel. I think > that this is moving in the right direction since you shouldn't need to > exhaustively know all the roles as long as you can attribute resources to > them correctly. I can imagine that in future you could have an external > identity service that signs a binding between frameworks and roles that > Mesos can verify. > > > We have discussed in the previous session, append the discussion history > as > > below. Let's have a further discussion to choose a better solution > between > > them, any comments and feedbacks would be very welcome! > > > > ----- Original message ----- > > From: Yong Qiao Wang/China/IBM > > To: [email protected] > > Cc: [email protected], [email protected], Qian AZ > Zhang/China/IBM@IBMCN, > > [email protected], [email protected] > > Subject: Re: Dynamic vs. implicit roles > > Date: Tue, Dec 1, 2015 10:27 AM > > > > Thanks Neil. > > > >> With implicit roles, that would involve: > >> > >> 1. Choosing a role name > >> 2. Configuring weights, ACLs, and quotas for the role. > >> 3. Configuring applications/frameworks to register using that role. > > > > [Yong Qiao] If applications/frameworks do not follow your rules, and > > register with another role, then how to prevent? and do we will still > > create this undesirable role in Mesos? Maybe we can only relay on ACLs to > > avoid this, but according to my understanding, ACLs is not required in > > Mesos. In addition, I am not sure whether it is make sence to use ACLs > for > > role validation. > > > > Regards! > > *Yong Qiao Wang* > > > > Re: Dynamic vs. implicit roles > > 6:55 AM > > Neil Conway to me (cc), Yong Feng, Qian AZ Zhang, [email protected], > Adam > > Bordelon > > Show more > > Hi Yong, > > > > Thanks for your feedback. > > > > On Mon, Nov 30, 2015 at 2:36 PM, Yong Feng <[email protected]> wrote: > > > > 2. MESOS-3988 treats it as part of framework API. I assume that the new > > created implicit role only has a kind of default resource plan like > > weight/quota. > > > > > > This is not the case: if you configure a weight/quota/ACL for a role, > that > > configuration will be used whenever a framework tries to register using > > that role. So you can configure a non-default weight/quota/etc. for an > > implicit role just as you would with a statically or dynamically > configured > > role. > > > > We still rely on management API such as quota management to further > > configure the resource plan. > > > > For the use case that in a company/organization, all resources are > > allocated according to pre-defined budget plan. We will need admin to > > create role, configure resource plan before launching application. > > Mesos-3988 does not help as the resource plan should be configured before > > application is running. > > > > > > I would phrase it as: in both designs, the admin will configure a > "resource > > plan". With dynamic roles, that would involve: > > > > 1. Choosing a role name > > 2. Creating a role with that name > > 3. Configuring weights, ACLs, and quotas for the role. > > 4. Configuring applications/frameworks to register using that role. > > > > With implicit roles, that would involve: > > > > 1. Choosing a role name > > 2. Configuring weights, ACLs, and quotas for the role. > > 3. Configuring applications/frameworks to register using that role. > > > > i.e., implicit roles are equivalent to dynamic roles, but slightly > simpler. > > If you see situations in which dynamic roles would allow you to do > > something that implicit roles would not, please let me know -- I'm not > > aware of any situations myself. > > > > For the use case that in a cloud environment, user would like to launch > app > > for a certain service level agreement. It does not make much sense to > > create a role for the application only and in advance. We could simply > > create a role for the "service level agreement", and then ask application > > register with the role. Regarding to allocating resources among > frameworks > > within the same service level agreement, we already have object of > > "framework" which is used as an entity when Mesos allocate resources. > > > > > > Sorry, I didn't quite follow what you mean here. > > > > Thanks, > > Neil > > > > To: Adam Bordelon <[email protected]> > > From: Yong Feng/Markham/IBM > > Date: 12/01/2015 06:36AM > > Cc: Benjamin Hindman <[email protected]>, Neil Conway < > [email protected]>, > > Yong Qiao Wang <[email protected]>, Qian AZ Zhang <[email protected]> > > Subject: Re: Dynamic vs. implicit roles > > > > We'd better to move it into dev@list. Just try to show my two cents in > this > > tread > > > > I see the main difference between the two proposals is that > > > > 1. MESOS-3177 introduces management API for role life cycle management > and > > in future for how to plan resource among tenants. We usually call > "planning > > resource among tenants" as resource plan. The quota management actually > > also belongs to it. > > 2. MESOS-3988 treats it as part of framework API. I assume that the new > > created implicit role only has a kind of default resource plan like > > weight/quota. We still rely on management API such as quota management to > > further configure the resource plan. > > > > For the use case that in a company/organization, all resources are > > allocated according to pre-defined budget plan. We will need admin to > > create role, configure resource plan before launching application. > > Mesos-3988 does not help as the resource plan should be configured before > > application is running. > > > > For the use case that in a cloud environment, user would like to launch > app > > for a certain service level agreement. It does not make much sense to > > create a role for the application only and in advance. We could simply > > create a role for the "service level agreement", and then ask application > > register with the role. Regarding to allocating resources among > frameworks > > within the same service level agreement, we already have object of > > "framework" which is used as an entity when Mesos allocate resources. > > > > So basically I did not see a strong use case Mesos-3988 could resolve > > while MESOS-3177 does not or need extra efforts. However I do see the use > > cases Mesos-3988 cannot resolve. > > > > Thanks, > > > > Yong > > > > [image: Inactive hide details for Adam Bordelon ---11/30/2015 03:26:55 > > AM---- In the implicit roles model, new roles are "created" when]Adam > > Bordelon ---11/30/2015 03:26:55 AM---- In the implicit roles model, new > > roles are "created" when a framework successfully registers under > > > > From: Adam Bordelon <[email protected]> > > To: Yong Qiao Wang <[email protected]> > > Cc: Neil Conway <[email protected]>, Benjamin Hindman < > [email protected]>, > > Qian AZ Zhang <[email protected]>, Yong Feng/Markham/IBM@IBMCA > > Date: 11/30/2015 03:26 AM > > Subject: Re: Dynamic vs. implicit roles > > ------------------------------ > > > > > > > > - In the implicit roles model, new roles are "created" when a framework > > successfully registers under that role. Other actions like creating a > > reservation/volume or setting a weight/quota implicitly "create" or > "name" > > a role, but it isn't active until there's a framework registered. > > - A role exists as long as it still has any registered frameworks that > > haven't timed out yet. Even then, it isn't really active in the allocator > > unless at least one of its frameworks is active, even if there are > > reservations/volumes/quota/weights associated. > > - Implicit roles don't need to be persisted, if any role is allowed. If > > there are ACLs restricting the set of allowed roles, then those ACLs will > > have to be stored statically in the master's --acls flag, or eventually > in > > the replicated log when we have dynamic ACLs. > > - Typos are unfortunate, but we have to trust that the operator can be > > consistent when configuring ACLs, weights, quota, etc. If we need an > > explicit role whitelist, then we can use ACLs to express that only > certain > > roles are allowed. > > > > Let's take future conversations onto the dev@ list so we can get others > > involved. > > > > On Sun, Nov 29, 2015 at 9:59 PM, Yong Qiao Wang <*[email protected]* > > <[email protected]>> wrote: > > > > In addition, Dynamic roles/weights(MESOS-3177) proposes a simplified > and > > centralized management(Creating/Removing/Updating/Persisting) for role > life > > cycle. If we propose to use "Implicit Roles(MESOS-3988)" to replace > > "Dynamic Roles(MESOS-3177)", then I want to know how we will cover the > same > > functions in Implicit Roles, for example: > > > > - When create a role in Mesos? > > - When delete a role in Mesos? > > - How to persist roles in replicated log? > > - How to avoid the typos? For example, role typos when framework > > register, when configure ACLs, when configure weight, etc. > > > > Neil, could you also help to clarify above concerns in your design doc? > > It is important to help us to make a right decision between Dynamic > Roles > > and Implicit Roles. > > > > Thanks! > > *Yong Qiao Wang* > > > > > > ----- Original message ----- > > From: Yong Qiao Wang/China/IBM > > To: *[email protected]* <[email protected]> > > Cc: *[email protected]* <[email protected]>, *[email protected]* > > <[email protected]> > > Subject: Re: Dynamic vs. implicit roles > > Date: Sat, Nov 28, 2015 10:07 PM > > > > Hi Neil, > > > > Thanks a million for your proposals, sorry for being so unresponsive > > lately, I have been taking vacation. > > > > I have read the design doc of implicit roles, one main commnet as > > below: > > > > Per my understanding, in Mesos, the role which is used to determine > > what resources frameworks can use, and the total number of roles > affects > > each role's fair share of the Mesos cluster, but in the proposal of > > implicit roles, framework can register with all possible role, so > how to > > guarantee this? > > > > Regards! > > *Yong Qiao Wang* > > > > > > ----- Original message ----- > > From: Neil Conway <*[email protected]* <[email protected]>> > > To: Yong Qiao Wang/China/IBM@IBMCN > > Cc: Benjamin Hindman <*[email protected]* <[email protected] > >>, > > Adam Bordelon <*[email protected]* <[email protected]>> > > Subject: Dynamic vs. implicit roles > > Date: Thu, Nov 26, 2015 7:36 AM > > > > Hi Yong, > > > > We've both been looking at how to improve role management in > > Mesos, so > > I wanted to get in touch about the best way to move forward here. > > > > As you know, there are two proposals: > > > > (a) Dynamic roles (MESOS-3177): roles are stored in the registry > > and > > added/deleted/removed via HTTP endpoints > > > > (b) Implicit roles (MESOS-3988): any role will be allowed, > subject > > to > > the ACL/authorization system. In a sense, "all roles" exist, so > > there > > is no need to store the set of legal roles or provide endpoints > to > > modify them. For more information on implicit roles, I posted a > > design > > doc here: > > * > https://docs.google.com/document/d/1SCFfrBd4edSY3bVCMrNJYMxIVllD0bHJuGmgG-4vCXA/edit?usp=sharing* > > < > https://docs.google.com/document/d/1SCFfrBd4edSY3bVCMrNJYMxIVllD0bHJuGmgG-4vCXA/edit?usp=sharing > > > > -- any feedback on the design doc would be very welcome! > > > > It seems we need to decide between implicit and dynamic roles -- > it > > wouldn't make sense to implement both. > > > > I'd like to suggest that we implement implicit roles, rather than > > dynamic roles. The reason is simplicity: with implicit roles, we > > can > > simply remove the list of "legal" roles, and instead rely on ACLs > > and > > the authorization mechanism to decide whether operations like > > registering a framework as a role or making a reservation are > > allowed. > > By removing the list of roles, we have one less piece of state we > > need > > to account for, store in the replicated log, provide HTTP > > endpoints to > > modify, etc. > > > > Let me know what you think! > > > > Once we have implicit roles, we will naturally want to support > (a) > > dynamic configuration of ACLs (b) dynamic configuration of > > weights. If > > you agree that implicit roles make sense, then I'd like to > propose > > that I implement implicit roles, while you can focus on doing > > dynamic > > weights. After those are both built, we can decide how to build > > dynamic ACLs. > > > > Neil > >
