opened the following to track the discussion https://github.com/kubernetes/kubernetes/issues/47326
On Saturday, June 10, 2017 at 1:42:39 AM UTC-7, Mayank wrote: > > Pinging again. I would like to learn how others using Kubernetes are > separating different tenants using dynamic provisioning. Specifically if > people are using RBD provisioning , are they allocating a StorageClass and > hence a Ceph pool per customer ? What advantages or disadvantages do you > see with this approach ? > > What isolation mechanisms you use to make sure a PV of one customer is > never seen/used by another customer ? > > -Mayank > > On Tuesday, June 6, 2017 at 11:32:55 PM UTC-7, Mayank wrote: >> >> Adding Kubernetes-Users group to gather thinking from the community. >> >> >> On Tuesday, June 6, 2017 at 10:55:25 PM UTC-7, krma...@gmail.com wrote: >>> >>> Thanks all. The suggestion to use the reclaimPolicy of delete would not >>> work. As Jeff points out, in case of Stateful apps, we would like to not >>> delete the volumes immediately but keep them around for some time and >>> garbage collect them later. >>> >>> @David Could you elaborate on the inefficient utilization when you >>> mention the following ?"If your set of tenants is very static, i guess you >>> could have one StorageClass per tenant and only use "recycle" reclaim >>> policy (which seems to be what you're advocating). But this seems pretty >>> inefficient from a utilization standpoint, as you'd end up accumulating the >>> max number of PVs used by each tenant." >>> >>> @David we are interested in RBD to start with. We would also be doing >>> EBS Volumes as well. >>> >>> @Mike yes looks like encryption seems like a reasonable solution as long >>> as we can separate the encryption key per tenant and only the tenant has >>> access to its own encryption keys. EBS volumes are the only ones supporting >>> encryption. So for RBD , we are out of luck. Is there a general pattern of >>> doing encryption with external provisioning ? What about incorporating in >>> in-tree plugins ? >>> >>> @Clayton Doesn't OpenShift have use cases for keeping the volumes >>> around after the pods are gone ? Does OpenShift do any kind of multi >>> tenancy on top ? >>> >>> @Tim thanks. Not advocating that PV should or should not be namespaced >>> but just trying to understand the rationale for it. One thinking was if >>> PV's were dynamically provisioned in the namespace of the customer, that >>> might further limit their access. >>> >>> @Jeff i think your understanding of the problem is correct. But its >>> possible, i am solving the wrong problem and thats why i am seeking out for >>> help. >>> >>> >>> Overall i want a multi tenant model, where:- >>> -- accidentally its not possible for one tenant to mount a volume >>> created by another tenant >>> -- its more secure and attacks /compromise limit the surface area of >>> attacks and access to volumes >>> >>> In the absence of encryption if there is a Kubernetes native way of >>> ACL'ing the PV's that doesnt rely on the underlying storage implementation >>> would be great. >>> >>> >>> On Monday, June 5, 2017 at 9:17:49 AM UTC-7, Jeff Vance wrote: >>>> >>>> "... guarantee that a volume allocated to one customer can never be >>>> accidentally allocated/mounted/accessed by another customer" >>>> >>>> I don't see "delete" as the answer here since this deletes the actual >>>> data in the volume. What if the customer has legacy data or important data >>>> that lives beyond the pod(s) accessing it? Perhaps an "exclusive" >>>> accessMode might help? FSGroup IDs can control access and there's been >>>> talk >>>> about adding ACLs to PVs. Or, maybe I've misunderstood the question? >>>> >>>> jeff >>>> >>>> On Saturday, June 3, 2017 at 11:39:16 PM UTC-7, krma...@gmail.com >>>> wrote: >>>>> >>>>> My team is currently trying to enable Stateful Apps for our internal >>>>> customers. One requirement that keeps coming up is how to isolate PV's of >>>>> one internal customer from PV's of another internal customer. >>>>> >>>>> I see the following isolation mechanisms:- >>>>> - A PV when bound to a PVC(inside namespace A) cannot be bound to >>>>> another PVC(inside namespace B) unless the unbind happens and hence are >>>>> exclusive. >>>>> - When using StorageClass, A PV of certain class can only be bound to >>>>> PVC of the same class. So that means PVC(of class A) can only be bound to >>>>> PV(of class A). This allows a PV allocated to one customer to not >>>>> accidently get allocated to another customer) >>>>> >>>>> While the above isolation is good, its not enough(as i understand it). >>>>> In a multi -tenant environment we want mechanisms which can guarantee >>>>> that >>>>> a volume allocated to one customer can never be accidentally >>>>> allocated/mounted/accessed by another customer. >>>>> >>>>> What is Kubernetes recommendation , on how to achieve this isolation ? >>>>> >>>>> Few more questions:- >>>>> - Why are Persistent Volumes not namespaced ? >>>>> - Is one or more StorageClass'es per Customer a good multi tenancy >>>>> model ? What other recommendations we have ? >>>>> >>>>> >>>>> Would love to hear the general thinking and around this from both >>>>> developers and community >>>>> -Mayank >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.
