Sorry for the late reply as I have been away from a computer for a few days.
Today, I do not use Kubernetes in production, it is only in use for POC work
so that I can understand how to utilize the K8s platform and test if it
works well with Ignite. So far, Ignite seems to be playing nicely with
Kubernetes although I have not fully tested the performance, etc.
The POC environment is running Kubernetes 1.9.6 with Ignite Fabric 2.4. I am
using Statefulsets to make sure Ignite has some stickiness to the nodes in
case of failures, etc. I am also utilizing local storage on each of the K8s
nodes so that Ignite persistence will work and it does, very well in fact.
*Making it work*
The following link will give you everything you need to know to enable local
stoarage on Kubernetes:
- In order to use persistence, you have to have K8s version 1.9.3 and above.
This is because there are some "feature-gates" that you must enable to allow
for the persistence to operate properly. If your K8s version is >= 1.9.3 and
< 1.10.0 then you need "feature-gates:
your K8s version is >= 1.10.0 then you only need "feature-gates:
- Next you will need to create a StorageClass where "volumeBindingMode:
WaitForFirstConsumer". This allows us to bind to a storage class and wait
for it to be picked up by a volume claim before it can be used.
- Now we need a DaemonSet that monitors the disk that we want to use for
local storage and checks for mounts. When it sees a new mount, it will
create a PersistentVolume(PV) in K8s. Fortunately Quay has a docker image
that does everything we need,
"quay.io/external_storage/local-volume-provisioner". This is awesome because
we can add more storage or mounts or both and our DaemonSet will pick it up
- Now you just need to create a StatefulSet that uses an Ignite docker
image, has a "volumeClaimTemplates" section which points to a the
"volume.beta.kubernetes.io/storage-class" we specified earlier, and a
"volumeMounts" section which points to a mount point that was
auto-discovered by the DaemonSet above. When you start the StatefulSet, it
will create a PersistentVolumeClaim for the mount you specified and bind
that to the StatefulSet.
Bingo, you have persistent storage!
As to your request for contributing to the cause on external K8s discovery,
I would love to do so, but at this time I will have to respectfully decline
as I have many irons in the fire. I appreciate the gesture and hope to help
the community in the future!
I also hope this post helps everyone in their K8s Ignite adventures!
Sent from: http://apache-ignite-users.70518.x6.nabble.com/