Hey Kevin! The timing for looking into PetSets is perfect I think. Kubernetes upstream folks are thinking about Sticky IPs [16] (which are not essential for Infinispan but other projects might be really interested in this) and I asked some time ago about exposing PetSets to the outside world [17] (which is essential for accessing the cluster using Hot Rod client).
Please keep me in the loop. I'm really interested in this. Thanks Sebastian [16] https://github.com/kubernetes/kubernetes/issues/28969 [17] https://groups.google.com/forum/#!topic/kubernetes-dev/K-9KA_wMbmk On Sun, Aug 21, 2016 at 7:05 PM, Kevin Conner <kcon...@redhat.com> wrote: > Apologies, I’ve been travelling to London for the JDG meeting. > > On 19 Aug 2016, at 10:00, Sebastian Laskawiec <slask...@redhat.com> wrote: > > I've been playing with Kubernetes PetSets [1] for a while and I'd like > to share some thoughts. Before I dig in, let me give you some PetSets > highlights: > > • PetSets are alpha resources for managing stateful apps in > Kubernetes 1.3 (and OpenShift Origin 1.3). > > • Since this is an alpha resource, there are no guarantees about > backwards compatibility. Alpha resources can also be disabled in some > public cloud providers (you can control which API versions are accessible > [2]). > > • PetSets allows starting pods in sequence (not relevant for us, > but this is a killer feature for master-slave systems). > > • Each Pod has it's own unique entry in DNS, which makes discovery > very simple (I'll dig into that a bit later) > > • Volumes are always mounted to the same Pods, which is very > important in Cache Store scenarios when we restart pods (e.g. Rolling > Upgrades [3]). > > Thoughts and ideas after spending some time playing with this feature: > > • PetSets make discovery a lot easier. It's a combination of two > things - Headless Services [4] which create multiple A records in DNS and > predictable host names. Each Pod has it's own unique DNS entry following > pattern: {PetSetName}-{PodIndex}.{ServiceName} [5]. Here's an example of > an Infinispan PetSet deployed on my local cluster [6]. As you can see we > have all domain names and IPs from a single DNS query. > > • Maybe we could perform discovery using this mechanism? I'm aware > of DNS discovery implemented in KUBE_PING [7][8] but the code looks trivial > [9] so maybe it should be implement inside JGroups? @Bela - WDYT? > > • PetSets do not integrate well with OpenShift 'new-app' command. > In other words, our users will need to use provided yaml (or json) files to > create Infinispan cluster. It's not a show-stopper but it's a bit less > convenient than 'oc new-app'. > > • Since PetSets are alpha resources they need to be considered as > secondary way to deploy Infinispan on Kubernetes and OpenShift. > > • Finally, the persistent volumes - since a Pod always gets the > same volume, it would be safe to use any file-based cache store. > > If you'd like to play with PetSets on your local environment, here are > necessary yaml files [10]. > > PetSets are still in extremely early stages and I am part of a working > group that has recently formed to cover the introduction of this within the > products, including driving any necessary changes back upstream. There > will be Cloud Enablement involvement in this effort and any adoption of > petsets will likely go through the existing project. > > Kev > > -- > JBoss by Red Hat > >
_______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
