Hi Pubudu and Imesh, Thank for the feedback on this. I used rancher as an orchestration tool with Cattle as a cluster manager.
I was able to do that using the AWS metadata API just as the same approach of Pubudu. I did the changes at this local PR. [1] <https://github.com/ChamaraPhilipsuom/docker-apim/pull/1>. (This may not be the best way to do this.) AFAIK there is no simple way to inject dynamic values into axis2.xml as we do with user-mgt.xml or api-manager.xml using JVM variables. Just using the ${variable.name} didn't work with axis2.xml. Hence had to use placeholders and replace those placeholders using the OS environment variables which will be resolved at orchestration. I saw property values are using placeholders fro ${httpsPort} and ${hostName}, but not for parameters. Kindly let me know if there is another way to use JVM environment variables in axis2.xml just as we do with user-mgt.xml or api-manager.xml. Now I have am able to deploy the API manager with pattern-3 in a fully automated manner. With the above changes, I am able to select whether the clustering should be enabled, which clustering scheme we should use, members set if "wka" is used or the AWS settings, if "aws" clustering is used etc, at the time of deployment and the rest, will be handled automatically. I think it is good if we can have that kind of control when we move towards continuous delivery and continuous integration. I would like to send a PR for these if we can come to conclusion on the best way of doing this. Another problem I am facing at the moment is fronting the clusters with ELBs. We have separate ELBs for separate clusters of components. (I.E One ELB for stores, one for publishers, one for keymanagers etc.) At the moment this doesn't work since I am not using sticky sessions with JSESSIONID. (I have done WSO2 IS clustering with sticky sessions enabled.) Is there any way that we can replicate HTTP sessions? Found this article [2] <http://wso2.com/library/articles/2012/10/getting-full-features-apache-tomcat-supported-wso2-application-server/#clustering> which does it with webapps. But not sure how we can do same with publisher, store, gateway worker, keymanager, traffic-manager and analytics cluster. Highly appreciate any suggestions to proxy the clusters with AWS ELBs. [1] https://github.com/ChamaraPhilipsuom/docker-apim/pull/1 [2] http://wso2.com/library/articles/2012/10/getting-full-features-apache-tomcat-supported-wso2-application-server/#clustering Thanks & Regards On Tue, Apr 25, 2017 at 9:57 AM, Imesh Gunaratne <im...@wso2.com> wrote: > Hi Chamara, > > It's nice to see your interest in running API-M on Docker on AWS. > > Are you running this deployment with ECS or did you start two VMs, > installed Docker on them and started containers in each host manually? > > If you are following the latter, you can send the host IP to the > containers via an environment variable as you would know the host that the > containers will be started. If containers are scheduled automatically, > using the metadata API for this purpose would be fine. > > For a production deployment, it might be more productive to use a > container cluster manager such as ECS, Kubernetes, DC/OS or Docker Swarm as > they provide basic deployment features such as networking, load balancing, > container scheduling, auto-healing, auto-scaling, etc OOB. > > Thanks > > On Mon, Apr 24, 2017 at 5:45 PM, Chamara Philips <chcphilips....@gmail.com > > wrote: > >> Hi Pubudu, >> Thank you very much for the insight. The problem is exactly that. We are >> using the container name which will eventually resolved to the conteiner >> ip. The problem is getting the IP of EC2 instance since those instances are >> dynamically started. The aprroach going to use is using the AWS metadata >> api, to findout the host ip in which the container is running. >> >> I there any other way to discover the host IP dynamically in the AWS >> environment? >> >> I have mapped container ports to EC2 instance 5701:5701. Hope it works. >> >> Thanks & Regards >> >> On Mon, Apr 24, 2017 at 3:41 PM, Pubudu Gunatilaka <pubu...@wso2.com> >> wrote: >> >>> Hi Chamara, >>> >>> What is the value for localMemberHost? That value should be the public >>> IP of the EC2 instance which will be used by other containers (members) to >>> communicate. Based on the log, it looks like you have the container IP as >>> the localMemberHost and containers have failed to communicate with each >>> other. >>> >>> As you are running docker container within an EC2 instance, you need to >>> map container ports to the EC2 instance. >>> >>> If the clustering configured properly you should see the member joined >>> message. >>> >>> Thank you! >>> >>> On Mon, Apr 24, 2017 at 12:23 PM, Chamara Philips < >>> chcphilips....@gmail.com> wrote: >>> >>>> >>>> Hi all, >>>> >>>> I have tried deploying a API manager cluster in AWS EC-2 environment, >>>> using docker. I have used one EC-2 instance as one host which will deploy >>>> one container in each of them. All together there are 7 EC-2 instances used >>>> as hosts running a container for publisher,store, worker, manager, >>>> keymanager, traffic manager and analytics containers as described in >>>> pattern-3 >>>> <https://github.com/wso2/docker-apim/tree/master/docker-compose/pattern-3> >>>> . >>>> >>>> I have enabled hazelcast clustering and used "AWS" as the hazelcast >>>> clustering scheme. I have provided >>>> >>>> <parameter name="domain">wso2.pub.store.domain</parameter> >>>> <parameter name="accessKey">${aws.client.id}</parameter> >>>> <parameter name="secretKey">${aws.client.secret}</parameter> >>>> <parameter name="securityGroup">${aws.security.group}</parameter> >>>> <parameter name="region">${aws.security.region}</parameter> >>>> <parameter name="tagKey">${aws.tag.key}</parameter> >>>> <parameter name="tagValue">${aws.tag.value}</parameter> >>>> >>>> <parameter name="localMemberPort">5701</parameter> >>>> >>>> in axis2.xml files in worker-manager containers and publisher-store >>>> containers. (I checked the values applied correctly as expected above.) >>>> Above are the only changes I have done in axis2.xml >>>> >>>> I have added cyphers into catalina-server.xml also. Please find the >>>> catalina-server.xml in the attachment. Above parameters were required as >>>> explaind in this article >>>> <https://docs.wso2.com/display/CLUSTER44x/Setting+up+a+Cluster+in+AWS+Mode> >>>> . >>>> >>>> Yet the AWS cluster doesn't seems to be correctly configured. As an >>>> example in worker-manager cluster same log as below is shown in both worker >>>> and manager nodes. >>>> >>>> INFO - HazelcastClusteringAgent Cluster domain: wso2.am.domain >>>> INFO - HazelcastClusteringAgent Loading hazelcast configuration from >>>> axis2 clustering configuration >>>> INFO - HazelcastClusteringAgent Using aws based membership management >>>> scheme >>>> INFO - HazelcastClusteringAgent Hazelcast initialized in 12531ms >>>> INFO - HazelcastClusteringAgent Local member: [${member.Id}] - Host: >>>> *${container-ip}*, *Remote Host:null*, Port: 5701, HTTP:8280, >>>> HTTPS:8243, Domain: wso2.am.domain, Sub-domain:worker, Active:true >>>> INFO - HazelcastClusteringAgent Elected this member [${member.Id}] as >>>> the Coordinator node >>>> INFO - HazelcastClusteringAgent Cluster initialization completed >>>> >>>> Note that the host is resolved with container ip and the remote host is >>>> null. Is that the expected behaviour. >>>> >>>> AFAIK the expectation is to have the second node joined as a member. Is >>>> there any configuration that I am missing? >>>> -- >>>> -- >>>> Hareendra Chamara Philips (BSc.Eng(Hons)) >>>> Sysco acceleration, >>>> SyscoLabs(Pvt) Ltd >>>> Mobile : +94 (0) 767 184161 <94767184161> | +65 (9) 425 2874 >>>> <6594252874> >>>> >>> >>> >>> >>> -- >>> *Pubudu Gunatilaka* >>> Committer and PMC Member - Apache Stratos >>> Software Engineer >>> WSO2, Inc.: http://wso2.com >>> mobile : +94774078049 <%2B94772207163> >>> >>> >> >> >> -- >> -- >> Hareendra Chamara Philips (BSc.Eng(Hons)) >> Sysco acceleration, >> SyscoLabs(Pvt) Ltd >> Mobile : +94 (0) 767 184161 <94767184161> | +65 (9) 425 2874 <6594252874> >> > > > > -- > *Imesh Gunaratne* > WSO2 Inc: http://wso2.com > T: +94 11 214 5345 M: +94 77 374 2057 <+94%2077%20374%202057> > W: https://medium.com/@imesh TW: @imesh > lean. enterprise. middleware > > -- -- Hareendra Chamara Philips (BSc.Eng(Hons)) Sysco acceleration, SyscoLabs(Pvt) Ltd Mobile : +94 (0) 767 184161 <94767184161> | +65 (9) 425 2874 <6594252874>
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
