Hi, We need to implement this Mesos/Marathon membership scheme for specific Marathon version(s). I think it's better to start with the stable released version: 0.15.3 and provide provisions to plugin a client for a different Marathon API (in case previous one is incompatible).
I took the liberty of implementing this for Carbon 4.2.0 and Carbon 4.4.x versions. Since we are dropping native clustering support from Carbon 5.0.x on-wards this membership scheme will not be required for products based on Carbon 5. Note that this implementation is strictly for Mesos tasks running on Marathon framework and will not work with other Mesos frameworks. Therefore I think it is better to name this as MesosMarathonMembershipScheme (might have to tweak the name) so that it is not misleading to a user. Following is a list of parameters that I came up with to be passed on via axis2.xml to configure a WSO2 cluster on Mesos/Marathon. - MARATHON_ENDPOINT This is REST API endpoint to query Marathon tasks - MARATHON_APPLICATION_IDS Comma-separated list of Marathon application IDs. There can be multiple apps created for a particular deployment which need to be configured as a single cluster. For eg: APIM; publisher and store. Each profile will be deployed as separate Marathon apps. Marathon REST API supports SSL and Basic access authentication. Mesos/Marathon membership scheme should support this as well. - ENABLE_BASIC_AUTH - MARATHON_USERNAME - MARATHON_PASSWORD In current released version (0.15.3) there is no way to pass meta-data information to ports being created for each Marathon app. Therefore when membership scheme is adding members it has to rely on a predefined index in the port list to retrieve the host port created for Hazelcast service. Simple convention would be to take the 1st port defined. Since the order is preserved when passing port mapping information (container port => host port), membership scheme can accurately find the Hazelcast service port. But in latest Marathon release [2] (v1.0.0) they have addressed this limitation by introducing a "portDefinition" section to add meta-data to ports for service discovery purposes. [1] https://github.com/mesosphere/marathon/releases/tag/v0.15.3 [2] https://github.com/mesosphere/marathon/releases/tag/v1.0.0-RC1 Thanks. On Sat, Mar 26, 2016 at 11:16 AM, Imesh Gunaratne <[email protected]> wrote: > Hi All, > > Please refer [1] on the initial discussion on deploying WSO2 middleware on > Apache Mesos. It needs a Carbon Membership Scheme for automatically > dis-coverying the Hazelcast based cluster and allowing the instances to be > scaled in any order without breaking the cluster at any point. > > On high level a such Carbon Membership Scheme [2] would work as follows: > > - When a Carbon server is bootstrapped, the clustering agent will > initialize the membership scheme given in the clustering configuration in > the axis2.xml (relates to Carbon 4.2.x, 4.4.x). > - Then it will try to talk to an API and get the list of members > available in the given cluster. > - The above API response should include the hostname/ip address and > the Hazelcast port (might be a proxy port) of each member. > - Thereafter the membership scheme will initialize the Hazelcast > configuration by using the above member information and let Carbon create > the Hazelcast instance. > - According to this model each Carbon server in the cluster will get > connected to every other instance. As a result the cluster will be able to > autoscaled in any order. > > In Mesos, the Marathon REST API [3] can be used for the above purpose. > Once a Carbon server is deployed in Mesos via Marathon, it schedules tasks > for creating the containers. Each task would represent a container and > containers would get dynamic host ports allocated for each port it exposes. > > Using the below API method [4] those tasks can be listed: > > *GET /v2/apps/<application-id>/tasks* > > HTTP/1.1 200 OKContent-Type: application/jsonServer: > Jetty(8.y.z-SNAPSHOT)Transfer-Encoding: chunked > > { > "tasks": [ > { > "host": "agouti.local", > "id": "my-app_1-1396592790353", > "ports": [ > 31336, > 31337 > ], > "stagedAt": "2014-04-04T06:26:30.355Z", > "startedAt": "2014-04-04T06:26:30.860Z", > "version": "2014-04-04T06:26:23.051Z" > }, > { > "host": "agouti.local", > "id": "my-app_0-1396592784349", > "ports": [ > 31382, > 31383 > ], > "stagedAt": "2014-04-04T06:26:24.351Z", > "startedAt": "2014-04-04T06:26:24.919Z", > "version": "2014-04-04T06:26:23.051Z" > } > ]} > > > [1] [Architecture] Deploying WSO2 Middleware on Mesos > [2] > https://github.com/wso2/kubernetes-artifacts/tree/master/common/kubernetes-membership-scheme > [3] https://mesosphere.github.io/marathon/docs/rest-api.html > [4] > https://mesosphere.github.io/marathon/docs/rest-api.html#get-v2-apps-appid-tasks > > > Thanks > > -- > *Imesh Gunaratne* > Senior Technical Lead > WSO2 Inc: http://wso2.com > T: +94 11 214 5345 M: +94 77 374 2057 > W: http://imesh.io > Lean . Enterprise . Middleware > > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Akila Ravihansa Perera WSO2 Inc.; http://wso2.com/ Blog: http://ravihansa3000.blogspot.com
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
