Hi Lakmal IMHO we need to consider the option of having more coarse-grained components. If you look at the existing components and feature, there are too fine-grained. This is true even in the previous Stratos and Carbon code bases. Lets not make the same mistake again here. If we go ahead with fine-grained components then it will become a maintenance night-mare in the future.
So I would suggest few logical components and few logical features(i.e Installable Units). Thanks, Sameera. On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <[email protected]>wrote: > Hi all, > > I will start separate threads to discuss each item in detail. What I can > see milestone one should be, changing code to apache and re-factor the code > to new structure. After discuss in detail will prioritize and then finalize > the next milestone items. > > Current Stratos we use osgi model and will continue it. We will simplify > folder structure as below. > > > / > -Components > -- all osgi components > - Features (logically bundle components together) > -- adc > -- load balancer > -- auto scaller > -- manager > -- cloud controller > -- cli > -- cartridge agent > - Products (bundle several features and make a product) > - Service-stubs > -- all service stubs > - Tools > -- tools for create cartridges > > > Please share your thoughts. > > > thanks > > > On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana < > [email protected]> wrote: > >> +1 for idea of load balance within static members. It allows to have >> more openings and be more flexible. >> >> >> Thank You, >> Dharshana. >> >> >> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana >> <[email protected]>wrote: >> >>> >>> >>> >>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando < >>> [email protected]> wrote: >>> >>>> Hi Sanjeewa, >>>> >>>> >>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana < >>>> [email protected]> wrote: >>>> >>>>> >>>>> >>>>> >>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana <[email protected] >>>>> > wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> I hope all commiters / folks are in the mailing list now. Here I have >>>>>> listed below, some identified improvements, features that like to have in >>>>>> Stratos. >>>>>> >>>>>> Any other improvements/features are mostly welcome and will add those >>>>>> to the bellow list, then we can discuss and then will create milestones >>>>>> plan for releases. ( We will commit current code based soon after we got >>>>>> the git repository available) >>>>>> >>>>>> Before that if some one want to get quick overview of current >>>>>> release/architecture of Stratos please see [1] for my reason blog post or >>>>>> more detail [2] with current product documentation, which will migrate to >>>>>> Apache soon. Also we will organize some hangout sessions to explain >>>>>> current >>>>>> architecture and code based, and it may helpful other commierts to come >>>>>> up-to speed quickly. >>>>>> >>>>>> Here the list; >>>>>> >>>>>> - Re-factor current code to Apache >>>>>> >>>>>> First we will move current code based to apache and then start >>>>>> re-factoring to apache and make a release without adding any other >>>>>> improvement or feature. >>>>>> >>>>>> >>>>>> - IaaS Independent cartridge for Linux. >>>>>> >>>>>> With the current architecture when you create a cartridge its depend >>>>>> on the IaaS. For example If you create a cartridge in EC2, we have to >>>>>> make >>>>>> an AMI cartridge. Likewise you have to create cartridges (say PHP >>>>>> cartridge) for each and every IaaSs for support the IaaSs. As a solution >>>>>> we >>>>>> can introduce LXC based cartridges (which IaaS independent cartridges) >>>>>> and >>>>>> can be consider LXC is the default Stratos runtime container for Linux >>>>>> cartridges. Also this will provide more scallability to the PaaS (I will >>>>>> start separate mail thread with more detail on this) >>>>>> >>>>>> >>>>>> - Support Windows based cartridges. >>>>>> >>>>>> If IaaS support windows VM then we can have windows based cartridges. >>>>>> It will lead to have .net cartridges >>>>>> >>>>>> >>>>>> - Expand auto-scaling algorithm >>>>>> >>>>>> Currently auto scaling decision take based on looking at length of >>>>>> the http/https request queue. We have to expand this to consider CPU >>>>>> usage, >>>>>> memory utilization ..etc of the cartridges. >>>>>> >>>>>> Before we introduce que length based decision making mechanism we had >>>>> load average based decision making mechanism. Each server need to expose >>>>> some API(some service) to get load average of given server. Load >>>>> calculation can be different from one server to other. Load balancer will >>>>> do web service call to those end points and update load average values of >>>>> member list periodically. >>>>> >>>> >>>> We will separate out the load balancer logic and the auto-scaling logic >>>> soon. This would require some code level changes. But the vision is there. >>>> >>>> In brief, we should build APIs in Auto-scaler, so that the external >>>> components could provide/insert information that matters the decision of >>>> scaling up/down, depending on the algorithm used. >>>> >>>> Normally that service returns integer value(let say 0 to 10) and we >>>>> can get some idea about load from that. When it comes to traffic route we >>>>> will give high priority to server with minimum load and so on. So that >>>>> type >>>>> of implementation would be ideal for this case as well. Synapse member >>>>> selection algorithm can be configurable and we can add new algorithm for >>>>> this if need. >>>>> >>>> >>>> This is an extension point, one could leverage. >>>> >>>>> >>>>>> - TCP level load balance. >>>>>> >>>>>> Current we only doing http/https based load balancing but need to >>>>>> expand this to do TCP level load balancing. >>>>>> >>>>>> This would be again nice feature IMO. Also i think we need >>>>> considerable amount of synapse transport level improvements to implement >>>>> this. WDYT? >>>>> >>>> >>>> We may require to write a TCP load balance endpoint, if one is not >>>> already available (I couldn't find any reference). >>>> >>>>> >>>>>> - Health monitoring >>>>>> >>>>>> Current release we haven't much in the cartridge heath monitoring but >>>>>> its very important and need to have in the future release. >>>>>> >>>>>> If we implemented above mentioned load average calculation mechanism >>>>> then we can do this easily. >>>>> >>>> >>>> Yes, health monitor would ideally consider the load average of the >>>> Cartridge instances, memory consumption etc. and transit the state >>>> according to a pre-defined curriculum, IMO. >>>> >>>> We can call that end point and get whatever we need. Also it would be >>>>> ideal if we can have some simple user interface for load balancer to view >>>>> active services and subscribed nodes and their status. Or else we can use >>>>> cluster message and set status of server as member attribute. >>>>> >>>> >>>>> Also i would like to suggest static load balance endpoint support for >>>>> load balancer(as a suggestion). Let say some user have fixed set of back >>>>> end servers and need to route traffic to them through load balancer. That >>>>> would be useful feature and we can consider it when we design milestone >>>>> plan. >>>>> >>>> >>>> I'm afraid this won't be a use-case when you consider Apache Stratos. >>>> Apache Stratos is a fully dynamic environment i.e. all the clusters get >>>> build, as and when required. We don't have a concept call static endpoints >>>> in the context of Stratos. >>>> >>> Yes i can understand. Thought it would be more useful if someone try to >>> load balance within static members. >>> >>>> >>>> Sample configuration would be as follows(It tells everything about >>>>> implementation). Sometimes back me and azeez had offline discussion about >>>>> this. >>>>> >>>>> TestService { >>>>> hosts server.test.com; >>>>> servers { >>>>> server1 { >>>>> ip = 192.0.0.1; >>>>> http = 80; >>>>> https = 443; >>>>> } >>>>> >>>>> server2 { >>>>> ip = 192.0.0.2; >>>>> http = 80; >>>>> https = 443; >>>>> } >>>>> } >>>>> } >>>>> >>>>>> >>>>>> - Improving ADC (Artifact Distribution Coordinator) >>>>>> >>>>>> Current release we only support git based artifact distribution and >>>>>> need to provide more scalable deployment options >>>>>> >>>>>> >>>>>> - Light weight cartridge agent >>>>>> >>>>>> Current load balancer used tribes based clustering. To support tribes >>>>>> clustering we have to use java based cartridge agent and it will put some >>>>>> unnecessary heavy load to non java cartridges like PHP. If we can moved >>>>>> to >>>>>> Hazelcast Clustering we can have python based light weight cartridge >>>>>> agent >>>>>> across all type of cartridges. >>>>>> >>>>>> >>>>>> - Adding more and more cartridges >>>>>> >>>>>> We can have more and more cartridges like Python, Ruby, Node.js, >>>>>> Mongo etc. >>>>>> >>>>>> >>>>>> thanks >>>>>> >>>>>> [1] >>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html >>>>>> [2] >>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation >>>>>> >>>>>> >>>>>> -- >>>>>> Lakmal Warusawithana >>>>>> Software Architect; WSO2 Inc. >>>>>> Mobile : +94714289692 >>>>>> Blog : http://lakmalsview.blogspot.com/ >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> *Sanjeewa Malalgoda >>>>> **B.Sc. Engineering(Hons) >>>>> Dip. in Com.Sc. >>>>> AMIESL , MIACSIT, CCNA >>>>> >>>>> * >>>>> Mobile +94713068779 >>>>> >>>>> http://sanjeewamalalgoda.blogspot.com/ >>>>> >>>> >>>> >>>> >>>> -- >>>> Best Regards, >>>> Nirmal >>>> >>>> C.S.Nirmal J. Fernando >>>> Senior Software Engineer, >>>> WSO2 Inc. >>>> >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>> >>> >>> Thanks, >>> -- >>> >>> *Sanjeewa Malalgoda** >>> >>> * >>> Mobile +94713068779 >>> >>> http://sanjeewamalalgoda.blogspot.com/ >>> >> >> >> >> -- >> ................................................... >> Dharshana Kasun Warusavitharana >> >> >> >> > > > -- > Lakmal Warusawithana > Software Architect; WSO2 Inc. > Mobile : +94714289692 > Blog : http://lakmalsview.blogspot.com/ > > -- Sameera Jayasoma blog: http://sameera.adahas.org twitter: https://twitter.com/sameerajayasoma flickr: http://www.flickr.com/photos/sameera-jayasoma/
