Hi Martin, Sorry for the late response.
On Sun, Oct 12, 2014 at 11:25 AM, Reka Thirunavukkarasu <[email protected]> wrote: > Hi Martin, > > On Fri, Oct 10, 2014 at 5:40 AM, Martin Eppel (meppel) <[email protected]> > wrote: > >> Hi Reka, >> >> >> >> I did some startup order / dependency testing and run into the issue that >> when a subgroup has no members listed a Null pointer exception is thrown in >> PaserUtils.java, line 127. Issue is that the >> >> GroupContext [] … in groupContext.getGroupContexts(); is never being set. >> I attached the application / group configuration jsons with which I run >> into the issue. Let me know if I made some configuration errors (startup >> dependencies ?) >> > > We have fixed the issue faced in DependencyBuilder. Now that startupOder > is getting assigned to DependencyTree properly. > >> >> >> Thanks >> >> >> >> Martin >> >> >> >> java.lang.NullPointerException >> >> at >> org.apache.stratos.cloud.controller.application.parser.ParserUtils.getAliasForGroupName(ParserUtils.java:127) >> > Could not reproduce this error yet. Are you still getting this? > > I'm not sure on this NPE. We will go through your definition and update on > it.. > > Thanks, > Reka > >> >> >> Thanks >> >> >> >> Martin >> >> *From:* Martin Eppel (meppel) >> *Sent:* Thursday, October 09, 2014 9:32 AM >> *To:* Reka Thirunavukkarasu >> *Cc:* [email protected]; Isuru Haththotuwa ([email protected]) >> *Subject:* RE: DependencyBuilder / StartupOrder clarification >> >> >> >> Great, thanks – sure will continue the testing, >> >> >> >> Regards >> >> >> >> Martin >> >> >> >> *From:* Reka Thirunavukkarasu [mailto:[email protected] <[email protected]>] >> *Sent:* Thursday, October 09, 2014 6:56 AM >> *To:* Martin Eppel (meppel) >> *Cc:* [email protected]; Isuru Haththotuwa ([email protected]) >> *Subject:* Re: DependencyBuilder / StartupOrder clarification >> >> >> >> Hi Martin, >> >> >> >> On Thu, Oct 9, 2014 at 11:40 AM, Reka Thirunavukkarasu <[email protected]> >> wrote: >> >> Thanks Martin for the changes... >> >> >> >> On Thu, Oct 9, 2014 at 6:27 AM, Martin Eppel (meppel) <[email protected]> >> wrote: >> >> Hi Reka, >> >> >> >> I adjusted the remaining code to the new scheme and also modified the >> code in the DependencyBuilder (and checked it in) but I am not sure if it >> is correct, again, since I am not clear on the use of “*private >> List<String> startList;*” , please take a look and let me know how it >> is supposed to work. >> >> >> >> >> >> >> >> Also, there is still an issue the the startupOrder not being properly set >> in the DependencyBuilder, will look into it tomorrow, >> >> Sure..I'm in the process of testing the flow..Will update on it.. >> >> >> >> We had to modify DependebcyBuilder to take the start up orders from the >> list. And i had to add back the other conditional blocks to make the >> dependency builder working. >> >> >> >> I have fixed issues faced in Topology, DependencyBuilder and in Monitors. >> Now that the hierarchical monitors are being registerd without any issues >> for the first cut. We need to bring one cluster active and then continue >> with the rest of the testing. Can you also continue testing, if you get a >> chance? >> >> >> >> Thanks, >> >> Reka >> >> >> >> Thanks >> >> >> >> Martin >> >> >> >> *From:* Martin Eppel (meppel) >> *Sent:* Wednesday, October 08, 2014 12:14 PM >> *To:* Reka Thirunavukkarasu >> *Cc:* Isuru Haththotuwa ([email protected]); [email protected] >> *Subject:* DependencyBuilder / StartupOrder clarification >> >> >> >> Hi Reka, >> >> >> >> I need some clarification on how the DependencyBuilder is supposed to >> work (while I am trying to replace the code with the new scheme to >> represent the startup order). >> >> >> >> In the class StartupOrder.java we maintain a structure “*private >> List<String> startList;*” which is being used later in the “ >> *DependencyBuilder.java*”, see code snipplet below. However, I can’t >> find anywhere in the code that the “startList” is initialized or set >> (getStartList(…) being invoked). I am not really sure what the intention is >> how this code is supposed to work ? >> >> >> >> Thanks >> >> >> >> Martin >> >> >> >> DependencyBuilder.java >> >> >> >> *Set<StartupOrder> startupOrderSet = dependencyOrder.getStartupOrders();* >> >> * ApplicationContext foundContext = null;* >> >> * for (StartupOrder startupOrder : startupOrderSet) {* >> >> * foundContext = null;* >> >> * for (String start : startupOrder.getStartList()) {* >> >> * ApplicationContext applicationContext = >> ApplicationContextFactory.* >> >> * getApplicationContext(start, >> component, dependencyTree);* >> >> * String id = applicationContext.getId(); //TODO >> change the id* >> >> >> >> * ApplicationContext existingApplicationContext =* >> >> * >> dependencyTree.findApplicationContextWithId(id);* >> >> * if (existingApplicationContext == null) {* >> >> * if (foundContext != null) {* >> >> * //appending the start up order to existing >> group/cluster* >> >> * >> foundContext.addApplicationContext(applicationContext);* >> >> * if (log.isDebugEnabled()) {* >> >> * log.debug("Found an existing >> [dependency] " + foundContext.getId() +* >> >> * " and adding the [dependency] " >> + id + " as the child");* >> >> * }* >> >> * } else {* >> >> * //adding list of startup order to the >> dependency tree* >> >> * >> dependencyTree.addApplicationContext(applicationContext);* >> >> * }* >> >> * } else {* >> >> * if (foundContext == null) {* >> >> * //assigning the found context to the later >> use.* >> >> * foundContext = existingApplicationContext;* >> >> * if (log.isDebugEnabled()) {* >> >> * log.debug("Found an existing >> [dependency] " + id + " and setting it " +* >> >> * "for the next dependency to >> follow");* >> >> * }* >> >> * } else {* >> >> * //TODO Throw exception, since another same >> start order already found* >> >> * log.warn("Startup order is not consistent. >> It contains the group/cluster " +* >> >> * "which has been used more than one >> in another startup order");* >> >> * }* >> >> >> >> * }* >> >> >> >> * }* >> >> >> >> * }* >> >> >> >> *From:* Reka Thirunavukkarasu [mailto:[email protected] <[email protected]>] >> *Sent:* Wednesday, October 08, 2014 9:00 AM >> *To:* Martin Eppel (meppel) >> *Cc:* Isuru Haththotuwa ([email protected]); [email protected] >> *Subject:* Re: [Grouping][Part-2] Composite Application Dependency Tree >> building based in Autoscaler >> >> >> >> Hi Martin, >> >> >> >> On Wed, Oct 8, 2014 at 3:56 AM, Martin Eppel (meppel) <[email protected]> >> wrote: >> >> Hi Reka, >> >> >> >> Just wanted to clarify, the changes suggested below should completely >> replace the previous structure (StartupOrder with before, after), not only >> in the json definition but also in all subsequent object models, correct ? >> >> Yah..We need to change the json, application parser and the Topology. >> >> >> >> Btw, what triggered this change ? >> >> As i think, it is naming issue. If we change it to terminate, then it >> will be more consistent with the naming that we currently use in cloud >> controller. Or did you mean something else? >> >> >> >> Thanks, >> >> Reka >> >> >> >> Thanks >> >> >> >> Martin >> >> >> >> *From:* Reka Thirunavukkarasu [mailto:[email protected]] >> *Sent:* Monday, October 06, 2014 2:22 AM >> *To:* dev >> *Subject:* Re: [Grouping][Part-2] Composite Application Dependency Tree >> building based in Autoscaler >> >> >> >> Hi Imesh, >> >> >> >> On Mon, Oct 6, 2014 at 2:45 PM, Imesh Gunaratne <[email protected]> wrote: >> >> Hi Reka, >> >> >> >> I have a small concern on using the term "kill" in this scenario, I think >> it would be much more elegant if we call it something like "terminate". >> WDYT? >> >> +1. It's more appropriate to use terminate. Will change it to terminate. >> >> >> >> Thanks, >> >> Reka >> >> >> >> >> >> >> >> On Mon, Oct 6, 2014 at 11:44 AM, Reka Thirunavukkarasu <[email protected]> >> wrote: >> >> Hi all, >> >> >> >> I have implemented the dependency tree as mentioned in my mail earlier. >> It will return the immediate children for the start able dependencies. >> >> >> >> FYI: a composite application has postgresGroup, php, mysqlGroup, app >> server and esb as it's immediate children and their start up order is as >> mentioned in the mail earlier. >> >> >> >> "startupOrders": [ >> “postgresGroup, php", >> "sqlGroup, tomcat", >> "tomcat, apimanager", >> "tomcat, esb” >> ] >> >> >> >> So, if we look at the kill behaviour of this composite Application, it >> will be like below: >> >> >> >> *kill-none* : none of them will be returned >> >> >> >> *kill-all*: all the elements in that dependency tree will be returned >> >> For eg: if something happened to postgresGroup, then all the children >> of dependency tree would be returned as php, mysqlGroup, app server and esb >> will be get killed. >> >> >> >> *kill-dependent*: all the children of that particular node in the >> dependency tree will be returned. >> >> For eg: If something happened to mysqlGroup, then subsequently >> tomcat, app server and esb would be get killed. >> >> >> >> Question: in case when we get more than one dependencies to be killed, >> can we kill all of them in parallel or do we have to wait until it's >> dependent cluster/group got killed? >> >> >> >> Thanks, >> >> Reka >> >> >> >> On Thu, Oct 2, 2014 at 5:58 PM, Reka Thirunavukkarasu <[email protected]> >> wrote: >> >> Hi Martin, >> >> >> >> On Wed, Oct 1, 2014 at 11:03 PM, Martin Eppel (meppel) <[email protected]> >> wrote: >> >> Hi Reka, >> >> >> >> Are you suggesting to replace the current startupOrder definition with >> the one mentioned below ? >> >> >> >> "startupOrder" : [ >> >> { >> >> "start":"aa", >> >> "after":"bb" >> >> } >> >> ] >> >> >> >> Replaced with >> >> >> >> "startupOrders": [ >> >> "mypostgresGroup, myphp", >> >> "mysqlGroup, mytomcat", >> >> "mytomcat, myapimanager", >> >> "mytomcat, myesb" >> >> ] >> >> >> >> I have a couple of questions, >> >> >> >> 1. If we use the cartridge alias and the group alias in the group >> / application dependency definition how will it work when we auto scale >> groups ? My current understanding is that to get group scaling to work we >> would need 2 parameters – group name (==group.name) and group instance >> id (== group.alias), one static and one dynamic. So I would think we’ll >> have to define the application dependencies and group dependencies based on >> the name and not the alias, but, during run time we have to calculate the >> dependencies based on the alias. >> >> I think is important to make the distinction between group type (or >> name,) and group instance Id, without it we won’t be able to implement >> group scaling, wdyt ? >> >> Thanks for pointing this out..Yah..As you have mentioned, if we are to >> scale the groups by creating new groups, then we will be unable to use the >> groups alias in place of startuporders. But stratos is tightly coupled with >> subscription to cluster as one to one mapping and also, load balancer uses >> one to one mapping between cluster and hostname. So, if we are to bring up >> new clusters/groups, then things might get complicated in stratos. As i >> explained in the [part-1] discussion, we thought of achieving scale by >> group member and scale by group using constructing the deployment policy in >> a more advanced manner. I will start a separate thread on that. According >> to all of our opinion, we can decide on how to follow that up. >> >> IMHO the startupOrders in composite application and group definitions >> (json ) should look like >> >> "startupOrders": [ >> “postgresGroup, php", >> "sqlGroup, tomcat", >> "tomcat, apimanager", >> "tomcat, esb” >> ] >> >> while the runtime representation of the logical relationship model for >> each group or cartridge should use the corresponding aliases (or instance >> Id) so the monitor will reference the aliases (or instance Ids) while the >> json application / group definition will reference the group name (or type) >> and cartridge type to define the dependencies, WDYT ? >> >> 2. If for example a cartridge has multiple dependencies we would >> just add another line to the the startupOrders : >> >> e.g. postgresGroup depends on php and abc would be >> represented by: >> "startupOrders": [ >> "postgresGroup, php", >> "postgresGroup, “abc” >> …. >> ] >> >> >> >> >> >> Otherwise I think the proposal looks good, +1 >> >> >> >> Thanks, >> >> Reka >> >> >> >> Thanks >> >> >> >> Martin >> >> >> >> *From:* Reka Thirunavukkarasu [mailto:[email protected]] >> *Sent:* Wednesday, October 01, 2014 5:58 AM >> *To:* dev >> *Cc:* Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel); >> Udara Liyanage >> *Subject:* [Grouping][Part-2] Composite Application Dependency Tree >> building based in Autoscaler >> >> >> >> Hi >> >> >> >> As you aware, in the composite application we can define the depencies >> between groups/cartridges. Autoscaler's responsible is to parse this >> dependencies and build up a logical relationship model in order to handle >> the dependency information among the child nodes. As we have the >> hierarchical monitors in autoscaler, i propose to have dependencies >> information in each monitor that they aware of (the immediate child only). >> In that monitor, we need to identify the group/cartridge which can be >> started in parallel. So that a monitor can look at it's dependency and >> control it's immediate children based on that. Once all the children are >> active, it can pass the control to it's parent. For Eg: >> >> >> >> If we take the top level in Composite application which has mysqlGroup, >> postgresGroup, php, tomcat, apimanager and esb. If they have an alias >> saying my + cartridge/groupName then we can define the dependency >> information as follows: >> >> - myPhp depends on myPostgresGroup (means postgresGroup >> should be started before php) >> >> - myTomcat depends on myMysqlGroup >> >> - myApiManager depends on myTomcat >> >> - myEsb depends on myTomcat >> >> >> >> Like wise, groups will define their own dependency as well. >> >> >> >> In that way, we need to represent these dependency information as part of >> Composite Application definition/GroupDefinition. In order to represent >> this dependency information given above for Composite Application, i would >> suggest to have the following in Composite Application definition. >> >> >> >> "startupOrders": [ >> >> "mypostgresGroup, myphp", >> >> "mysqlGroup, mytomcat", >> >> "mytomcat, myapimanager", >> >> "mytomcat, myesb" >> >> ] >> >> >> >> You can use the same format in GroupDefinition to define dependencies in >> a group. >> >> >> >> As per the example, autoscaler will build a dependency tree for >> ApplicationMonitor as below in order to identify the parallel and dependent >> ones. So that Autoscaler will start up same level children monitors as in >> parallel. >> >> >> >> >> >> >> As above, ApplicationMonitor will start GroupMonitors of myPostgresGroup >> and myMysqlGroup in parallel. Once postgres becomes active, >> ApplicationMonitor will start ClusterMonitor for myPhp. Once >> myPostgresGroup becomes active, ApplicationMonitor will start the immediate >> child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start >> the myAppServer and myEsb in parallel. This will be applicable for >> GroupMonitors as well. They can look at their own dependency tree and will >> start their children. >> >> >> >> Please share your suggestions on the above model to handle the Dependency >> information of Composite Application in autoscaler. >> >> >> >> >> >> Thanks, >> >> Reka >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> Reka Thirunavukkarasu >> Senior Software Engineer, >> WSO2, Inc.:http://wso2.com, >> >> Mobile: +94776442007 >> >> >> >> >> >> >> >> >> >> -- >> >> Reka Thirunavukkarasu >> Senior Software Engineer, >> WSO2, Inc.:http://wso2.com, >> >> Mobile: +94776442007 >> >> >> >> >> >> >> >> -- >> >> Reka Thirunavukkarasu >> Senior Software Engineer, >> WSO2, Inc.:http://wso2.com, >> >> Mobile: +94776442007 >> >> >> >> >> >> >> >> >> -- >> >> Imesh Gunaratne >> >> >> >> Technical Lead, WSO2 >> >> Committer & PMC Member, Apache Stratos >> >> >> >> >> >> -- >> >> Reka Thirunavukkarasu >> Senior Software Engineer, >> WSO2, Inc.:http://wso2.com, >> >> Mobile: +94776442007 >> >> >> >> >> >> >> >> -- >> >> Reka Thirunavukkarasu >> Senior Software Engineer, >> WSO2, Inc.:http://wso2.com, >> >> Mobile: +94776442007 >> >> >> >> >> >> >> >> -- >> >> Reka Thirunavukkarasu >> Senior Software Engineer, >> WSO2, Inc.:http://wso2.com, >> >> Mobile: +94776442007 >> >> >> >> >> >> >> >> -- >> >> Reka Thirunavukkarasu >> Senior Software Engineer, >> WSO2, Inc.:http://wso2.com, >> >> Mobile: +94776442007 >> >> >> > > > > -- > Reka Thirunavukkarasu > Senior Software Engineer, > WSO2, Inc.:http://wso2.com, > Mobile: +94776442007 > > -- > <%2B94776442007> > Thanks and Regards, > > Isuru H. > <%2B94776442007> > +94 716 358 048 <%2B94776442007>* <http://wso2.com/>* > > > * <http://wso2.com/>* > > >
