Hi, To run WSO2 products directly on Kubernetes, we already have a membership scheme to discover members to support clustering [1]. However, when Kubernetes is deployed on top of Mesos, this approach will not work due to the following reasons:
- Mesos will be creating/destroying pods, not directly Kubernetes - If containers need to communicate with each other, needs to use host IP and the relevant mapped port in the host machine Therefore, to support member discovery when Kubernetes is deployed on top of Mesos, we extended the default Kubernetes Membership scheme and added the support to get the IP of the host node and the relevant port exposed by the host for each pod (Cluster Member), instead of using container IP and port. This is used to identify the members of the cluster and communicate within the same. The steps used to obtain the IP and mapped port in host for each pod are: 1. Read the Stratos cluster ids from the configuration, which should be in a single cluster in Hazlecast 2. Get the relevant Kubernetes Service (defined for clustering port) using the Stratos Topology, relevant to cluster ids read from configuration 3. Query the relevant pod names for each Kubernetes Service (using Kubernetes API) 4. Using the status [2] and annotation [3] of each pod, get the host IP and the host port (using Kubernetes API). 5. Use the host IP and port to initialize the Hazlecast member wdyt? [1]. https://github.com/wso2/kubernetes-artifacts/tree/master/common/kubernetes-membership-scheme <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fwso2%2Fkubernetes-artifacts%2Ftree%2Fmaster%2Fcommon%2Fkubernetes-membership-scheme&sa=D&sntz=1&usg=AFQjCNGR-0fSahNnC3EockiL9Ls9k532pw> [2]. "status": { "phase": "Running", "conditions": [ { "type": "Ready", "status": "True", "lastProbeTime": null, "lastTransitionTime": null } ], *"hostIP": "172.17.8.102",* "podIP": "10.244.27.4", } [3]. "annotations": { "applicationId": "wso2am-191-uacc-application-with-is", "cartridgeType": "wso2am-191-gw-manager", "clusterId": "wso2am-191-uacc-application-with-is.gateway-manager.wso2am-191-gw-manager.domain", "clusterInstanceId": "wso2am-191-uacc-application-with-is-1", "k8s.mesosphere.io/bindingHost": "10.46.12.55", "k8s.mesosphere.io/executorId": "6f8fdd082fc306fc_k8sm-executor", "k8s.mesosphere.io/offerId": "572751b9-9386-42ad-b0e1-79860d588e4f-O823906", "k8s.mesosphere.io/portName_TCP_http-9763": "1030", "k8s.mesosphere.io/portName_TCP_https-9443": "1029", "k8s.mesosphere.io/portName_TCP_tcp-22": "1031", *"k8s.mesosphere.io/portName_TCP_http-4000 <http://k8s.mesosphere.io/portName_TCP_http-4000>": "1032",* "k8s.mesosphere.io/port_TCP_22": "1031", "k8s.mesosphere.io/port_TCP_9443": "1029", "k8s.mesosphere.io/port_TCP_9763": "1030", * "k8s.mesosphere.io/port_TCP_4000 <http://k8s.mesosphere.io/port_TCP_4000>": "1032", * "k8s.mesosphere.io/slaveId": "e1d02d40-1496-42d1-8ad0-774dd0cb016e-S5", "k8s.mesosphere.io/taskId": "pod.22c48583-db19-11e5-a78c-005056ad6f59", "memberId": "wso2am-191-uacc-application-with-is.gateway-manager.wso2am-191-gw-manager.domain7cdce2a8-107e-411f-9943-a242fe04ee30" } -- Thanks and Regards, Isuru H. +94 716 358 048* <http://wso2.com/>*
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
