If we omit authz for admin endpoints like this, it would be great to disable them _by default_ until we have proper authorization in place. Otherwise operators will accidentally leave admin endpoints open for anyone to hit! :)
On Wed, Jul 15, 2015 at 12:07 PM, Alex Rukletsov <[email protected]> wrote: > Folks, > > We have updated the design doc [1] based on numerous comments on the v1. > > There is one substantial design question left: what entity should decide > whether a quota can be granted: the Master, an allocator, or a separate > "Quota Manager". I have described pros and cons of each in the doc, please > look inside for more information. > > Apart of that, I have made the following amendments: > * Clarified the absence of authz and how this mitigate that with firewall; > * Updated confusing naming: quota is a pair of guaranteed resource and > limit; > * Added a safety design principal; > * Updated section explaining granting quota decision; > * Updated QuotaInfo protobuf; > * Added an alternative quota implementation via dynamic reservations; > * Updated HTTP api to be more REST-like; > * Added more ideas to the wip Allocator section. > > Please have a look and check whether your concerns have been addressed. > > [1]: > > https://docs.google.com/document/d/16iRNmziasEjVOblYp5bbkeBZ7pnjNlaIzPQqMTHQ-9I/edit# > > On Fri, Jul 10, 2015 at 8:52 AM, Jörg Schad <[email protected]> wrote: > > > I would like to propose one alternative MVP proposal for the actual > quota > > implementation. > > > > Instead of making changes to the allocator we could have an allocator > > agnostic “Quota manager” which builds on top the existing dynamic > > reservations. > > > > Beyond MVP, we would still allow for allocator based implementations for > > more complex quota mechanism, but this quota manager could also offer > quota > > support for allocators which are not aware of quota (TBD). > > > > > > General Flow: > > > > - Quota manager leverages dynamic reservations to fulfill quota requests. > > It basically continuously tries to match the desired state (quota > requests) > > with the actual state (dynamic reservations). > > > > - it receives quota requests from master > > > > - using (approximate) slave usage information it selects target > slave/agent > > > > - requests to dynamic reservation (HTTP endpoint?), if fails retry > > > > - Master should notify QM of slave failures and reregistrations > > > > - in case of slave failures needs to request new reservation > > > > - in case of slave reregistrations might need to unreserve > > > > Required state of “Quota Manager” > > > > - current dynamic reservations > > > > - current granted quota > > > > - approximate slave available resources (in order to decide on which > slave > > to /reserve, but doesn’t matter if slightly out of sync -> in worst case > > /reserve fails and we have to retry) > > > > Failover > > > > - current granted quota are persisted in registry by master > > > > - current dynamic reservations are reconstructed from slave > reregistration > > > > Master needs to propagate information about slave failures to Quota > > Manager, as it might need to create new dynamic reservation in that case > > > > > > Advantages compared to allocator based implementation: > > > > - no need to change allocator(s) for MVP > > > > - quota support for external quota-agnostic allocators > > > > - re-using existing mechanisms > > > > - minimal implementation effort for MVP > > > > - almost free support for quota chunks (even in MVP) as dynamic > > reservations are per slave > > > > Disadvantages > > > > - allocator based quota implementation still needed for more elaborate > > implementations > > > > - as dynamic reservations do not account towards fair share, so wouldn’t > > quota based on this implementation. In my opinion this is not a real > > problem as a) we did not really define the semantics of quota and b) fair > > share is a allocator internal (i.e. DRF internal) notion so other > allocator > > implementations are free to do that differently anyhow. > > > > Looking forward to feedback! > > Jörg > > > > On Thu, Jul 9, 2015 at 11:52 PM, Tomás Senart <[email protected]> > wrote: > > > > > What about "Global Reservations"? > > > > > > On Thu, Jul 9, 2015 at 3:25 PM, Marco Massenzio <[email protected]> > > > wrote: > > > > > > > I've added my twocent in the doc - my vote goes for "Guaranteed > > > Allocation" > > > > - not as catchy as "Quota" (and will make classes' naming a > challenge!) > > > but > > > > maybe more helpful in the long-term. > > > > > > > > Anyone has a better suggestion, please do... I can't really say I'm > > > > super-excited by Guaranteed Allocation myself! > > > > > > > > *Marco Massenzio* > > > > *Distributed Systems Engineer* > > > > > > > > On Thu, Jul 9, 2015 at 1:48 AM, Alex Rukletsov <[email protected]> > > > > wrote: > > > > > > > > > And you're not the only one who were confused by the terminology! > One > > > of > > > > > the alternatives that didn't make it to the public doc was > > > "cluster-wide > > > > > dynamic reservations". The reason we preferred "quota" to " ... > > > > > reservation" is because the latter is already overloaded with > > meanings > > > in > > > > > Mesos world (static reservations, dynamic reservations). I have > hoped > > > the > > > > > Terminology section would have helped to avoid the confusion, but I > > see > > > > it > > > > > doesn't. We'll think about how we can solve the problem, we > > definitely > > > > > don't want to create one more "libprocess process represented as a > > > thread > > > > > in an OS process" ; ). > > > > > > > > > > I see your point regarding authorization, you're not alone here > > either > > > : > > > > ). > > > > > Some folks mentioned that the lack of authz is a blocker and will > > > prevent > > > > > them from upgrading the cluster. I would propose to treat MVP as > > > > > experimental feature: use it at your own risk or disable endpoints > > > > related > > > > > to quota and hence the entire feature. Does it make sense? > > > > > > > > > > On Wed, Jul 8, 2015 at 7:10 PM, James Peach <[email protected]> > > wrote: > > > > > > > > > > > > > > > > > > On Jul 4, 2015, at 3:15 AM, Alex Rukletsov < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > Folks, > > > > > > > > > > > > > > Jörg and I are working on adding *quota* support to Mesos. > Quota > > > can > > > > be > > > > > > > described as cluster-wide dynamic reservation. I would like to > > > share > > > > > the > > > > > > > design doc [1] to gather community feedback early in the design > > > > phase. > > > > > > > > > > > > The most confusing part of this document to me was the 'quota' > > > > > > terminology. Quotas normally refer to administrative limits (esp. > > > disk > > > > > > quotas with hard and soft limits), not reserving resources. Since > > > what > > > > > you > > > > > > are describing is an extension to the resource reservation > system, > > it > > > > > would > > > > > > be clearer if it was described in those terms. > > > > > > > > > > > > I was also concerned that access control / authorization is not > > > planned > > > > > > for the initial implementation. I think that if Mesos is to have > an > > > > > > authorization policy, it should be applied uniformly following > the > > > > > > principle of least surprise. > > > > > > > > > > > > > The doc is work in progress, especially the part related to > quota > > > > > support > > > > > > > in the allocator. We think we can start working on adding quota > > > > support > > > > > > to > > > > > > > Mesos Master while fleshing out the design for how quota is > > handled > > > > by > > > > > > the > > > > > > > built-in allocator. > > > > > > > > > > > > > > While working on the design, we faced some challenges and > design > > > > > > questions. > > > > > > > One of them is what decisions should be deferred to allocator > and > > > > what > > > > > > can > > > > > > > be decided by the Master. We elaborate on this in the doc. > > > > > > > > > > > > > > Looking forward to your feedback! > > > > > > > > > > > > > > [1]: > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/16iRNmziasEjVOblYp5bbkeBZ7pnjNlaIzPQqMTHQ-9I/edit?usp=sharing > > > > > > > > > > > > > > > > > > > > > > > > > > >
