Hi Osura, On Thu, Jul 21, 2016 at 11:15 AM, Osura Rathnayake <[email protected]> wrote:
> Hi Mentors, > > I will try using Puppet. > It wasn't a problem with log path/pattern, in fact I used the same log > path that I used last time. I believe it was a bug from Azure side, please > check the attached screenshots. > Shall we please have the meeting on Friday? > +1. How about 2.00 - 3.0.0 PM on Friday? > > thank you, > > On Wed, Jul 20, 2016 at 6:50 PM, Isuru Haththotuwa <[email protected]> > wrote: > >> Hi Osura, >> >> Shall we have a hangout on Thursday/ Friday to discuss and demonstrate >> the current progress of the project? Please let us know your preference. >> >> On Wed, Jul 20, 2016 at 3:35 PM, Imesh Gunaratne <[email protected]> wrote: >> >>> Hi Osura, >>> >>> On Mon, Jul 18, 2016 at 4:20 PM, Osura Rathnayake <[email protected]> >>> wrote: >>> >>>> Hi Mentors, >>>> >>>> This is the progress so far. >>>> >>>> *Auto scaling* >>>> >>>> >>>> Azure supports two paradigms of auto scaling, vertical and horizontal. >>>> Vertical scaling, also known as scale up and scale down, means increasing >>>> or decreasing virtual machine (VM) sizes in response to a workload. As I >>>> explained in one of my previous emails, vertical auto scaling is achieved >>>> through monitoring rules. We can set it to trigger an alarm when certain >>>> conditions are met and also as the action we can set up a runbook to scale >>>> up or down. I could successfully get VMs to scale up and down using this >>>> feature. >>>> >>> >>> We would use horizontal scaling in most scenarios. >>> >>> >>>> >>>> Horizontal scaling, also referred to as scale out and scale in, where >>>> the number of VMs is altered depending on the workload. Horizontal scaling >>>> is achieved through virtual machine scale sets (VMSS). One important thing >>>> about VMSS is that the VMs included should be of the same size and have the >>>> same OS image. All the VMs in the scale set are added to the load balancer, >>>> as a backend pool. Backend pool is a pool of VMs which share the traffic >>>> coming via the load balancer. We can add auto scale rules, as to when >>>> additional VMs should be added and removed, based on the conditions. I >>>> could test some auto scale rules and observed VMs getting added to the >>>> backend pool. But one problem is that when these VMs getting added, it’s a >>>> whole new VM. I’m trying to add custom made VMs which has a wso2 product >>>> installed and configured. >>>> >>> >>> Yes we would need a VM image with required WSO2 product and >>> pre-requisites to test this. At WSO2 we use Puppet >>> >>> for automating the installation process [1]. With Puppet we can use one >>> base VM image and start a VM for any WSO2 product. Puppet does the WSO2 >>> product installation at the VM startup. >>> >>>> >>>> Note: only a limited horizontal scaling features are supported in the >>>> azure portal yet so I’m using REST API to create and manage auto scaling >>>> features. >>>> >>>> >>>> *Centralized logging* >>>> >>>> >>>> I was able to get log to Log Analytics using custom logs. I have >>>> collected logs generated from 2 wso2 application servers. You only have to >>>> add respective VMs to the Log analytics and set the path where logs are >>>> located. This feature was under public preview, which might have been one >>>> reason why it didn’t work last time when I tried. >>>> >>> >>> Was the issue with the log file path/pattern? Did it work once you >>> remove that? >>> >>> [1] >>> https://github.com/wso2/puppet-modules/wiki/Use-WSO2-Puppet-Modules-in-puppet-master-agent-Environment >>> >>> Thanks >>> >>>> >>>> Thanks, >>>> >>>> On Mon, Jul 18, 2016 at 8:21 AM, Osura Rathnayake <[email protected]> >>>> wrote: >>>> >>>>> Hi Isuru, >>>>> >>>>> Please accept my apologies I have messed up names in my last email. >>>>> I'm not going to be available today due to an unavoidable circumstance so >>>>> can we please have the meeting on Wednesday? Extremely sorry if it made >>>>> any >>>>> inconvenience. I will update you with a detailed email today for sure. >>>>> >>>>> Thanks in advance, >>>>> >>>>> On Fri, Jul 15, 2016 at 4:04 PM, Isuru Haththotuwa <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Osura, >>>>>> >>>>>> Can you send a detailed updated on the current status? Shall we have >>>>>> a meeting on Monday to review the progress. >>>>>> >>>>>> On Tue, Jul 12, 2016 at 2:03 PM, Osura Rathnayake <[email protected] >>>>>> > wrote: >>>>>> >>>>>>> Hi Imesh, >>>>>>> >>>>>>> About dynamically adding members to the load balancer, I will cross >>>>>>> check it with auto-scaling. I couldn't look into that from auto-scaling >>>>>>> end >>>>>>> since I couldn't test it yet. >>>>>>> >>>>>>> about monitoring, yes we can do a POC on that. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> On Tue, Jul 12, 2016 at 12:53 PM, Imesh Gunaratne <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> On Tue, Jul 12, 2016 at 12:09 PM, Osura Rathnayake < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Imesh, >>>>>>>>> >>>>>>>>> About centralized logging, I'm trying to get logs to the Log >>>>>>>>> Analytics using few methods supported in azure. We can either parse >>>>>>>>> logs in >>>>>>>>> to syslogs and send to the Log Analytics or create custom logs >>>>>>>>> specifying >>>>>>>>> the logs location. As you said, logs shouldn't be in .txt extension, >>>>>>>>> I got >>>>>>>>> it clarified from a azure blog. Will update you soon after I could >>>>>>>>> resolve >>>>>>>>> it. >>>>>>>>> >>>>>>>>> Right, thanks for the update! >>>>>>>> >>>>>>>> >>>>>>>>> No you can't dynamically add VMs to the load balancer. Backend >>>>>>>>> pool, where all the VMs reside, should be predefined. >>>>>>>>> >>>>>>>> >>>>>>>> Technically that capability should be there. Otherwise we would >>>>>>>> not be able to autoscale a server cluster dynamically. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> you can auto-scale using scale sets(I'm still researching about >>>>>>>>> it), that's the equivalent of AWS auto scaling group . also you can >>>>>>>>> scale >>>>>>>>> up or down a VM if it exceeds a certain parameter like CPU usage, >>>>>>>>> using >>>>>>>>> monitoring rules. >>>>>>>>> >>>>>>>>> *Monitoring * >>>>>>>>> >>>>>>>>> >>>>>>>>> Azure has a native monitoring tool which involves collecting and >>>>>>>>> tracking metrics, analyzing log files, defining custom metrics and >>>>>>>>> logging >>>>>>>>> generated by specific applications or workloads running in Virtual >>>>>>>>> Machines. Azure represents monitored data in a graphical manner using >>>>>>>>> charts. Monitoring also facilitates triggering alarms when certain >>>>>>>>> conditions are met and also it can be configured to take actions on >>>>>>>>> the met >>>>>>>>> conditions. Monitoring is done by the Diagnostic Extension and it has >>>>>>>>> following capabilities. >>>>>>>>> >>>>>>>>> · Collects and uploads the system performance information >>>>>>>>> from the Linux VM to the user's storage table, including diagnostic >>>>>>>>> and >>>>>>>>> syslog information. >>>>>>>>> >>>>>>>>> · Enables users to customize the data metrics that will >>>>>>>>> be collected and uploaded. >>>>>>>>> >>>>>>>>> · Enables users to upload specified log files to a >>>>>>>>> designated storage table. >>>>>>>>> >>>>>>>>> Note: Azure storage tables are a non-relational, key-value-pair, >>>>>>>>> storage system suitable for storing massive amounts of unstructured >>>>>>>>> data. >>>>>>>>> >>>>>>>>> >>>>>>>>> We can add monitor rules so that when an alert triggers it >>>>>>>>> notifies the admins via email. Furthermore we can set to take >>>>>>>>> automated >>>>>>>>> actions. Azure automate actions by running runbooks. A runbook is a >>>>>>>>> set of >>>>>>>>> tasks that perform some automated process in Azure Automation. We can >>>>>>>>> create our own runbooks as well. Available runbooks include, >>>>>>>>> >>>>>>>>> · Restart VM >>>>>>>>> >>>>>>>>> · Stop VM >>>>>>>>> >>>>>>>>> · Remove VM >>>>>>>>> >>>>>>>>> · Scale up VM >>>>>>>>> >>>>>>>>> · Scale down VM >>>>>>>>> >>>>>>>>> When scaling up it sets the virtual machine to the next larger >>>>>>>>> size within the size group and when scaling down it sets the virtual >>>>>>>>> machine to the next smaller size within the size group. >>>>>>>>> >>>>>>>>> More about runbooks and automation [1] >>>>>>>>> >>>>>>>> >>>>>>>> Sounds good, will us be able to do a POC on this? >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> *Auto scaling * >>>>>>>>> >>>>>>>>> >>>>>>>>> Auto-scaling is the process of dynamically allocating the >>>>>>>>> resources required by an application to match performance >>>>>>>>> requirements. >>>>>>>>> Virtual machine scale sets are an Azure Compute resource you can use >>>>>>>>> to >>>>>>>>> deploy and manage a set of identical VMs. With all VMs configured the >>>>>>>>> same, >>>>>>>>> VM scale sets are designed to support true auto-scale no >>>>>>>>> pre-provisioning >>>>>>>>> of VMs is required – and as such makes it easier to build large-scale >>>>>>>>> services targeting big compute, big data, and containerized workloads >>>>>>>>> [2]. >>>>>>>>> >>>>>>>>> >>>>>>>>> Note: I couldn’t practically do this as my azure free account lets >>>>>>>>> me have only 4 cores and I have used all of them on my current >>>>>>>>> deployment. >>>>>>>>> I’m getting a new azure account from one of my friends in a day so >>>>>>>>> hopefully I will do this on it and update you. >>>>>>>>> >>>>>>>> >>>>>>>> Great! Thanks! >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> https://azure.microsoft.com/en-us/documentation/articles/automation-intro/ >>>>>>>>> >>>>>>>>> [2] >>>>>>>>> https://azure.microsoft.com/en-us/documentation/articles/virtual-machine-scale-sets-overview/ >>>>>>>>> >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Jul 11, 2016 at 9:54 AM, Imesh Gunaratne <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> On Thu, Jul 7, 2016 at 7:11 PM, Osura Rathnayake < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Mentors, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> In addition to refining the membership scheme code, I looked >>>>>>>>>>> into following features of Azure. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> Good findings Osura, please find few questions inline: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *Azure dynamic load balancing* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Azure has a native load balancer which is very easy to >>>>>>>>>>> configure. It’s a layer 4 (TCP, UDP) load balancer which helps to >>>>>>>>>>> spread >>>>>>>>>>> traffic among healthy virtual machines. Following are some key >>>>>>>>>>> terms you >>>>>>>>>>> need to know. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> Can members be dynamically added and removed to/from a load >>>>>>>>>> balancer? To check this we may need to explore how autoscaling >>>>>>>>>> works. On >>>>>>>>>> AWS this is handled with autoscaling groups [3] >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *Capturing Virtual Machine Images as templates* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Azure provides the feature of generalizing and capturing virtual >>>>>>>>>>> machines so that they can be used as templates. This is very useful >>>>>>>>>>> and >>>>>>>>>>> time saving when the production environment has many instances of >>>>>>>>>>> the same >>>>>>>>>>> kind of virtual machine. When the virtual machine is being >>>>>>>>>>> generalized all >>>>>>>>>>> the data in user directories are erased so better to have wso2 >>>>>>>>>>> product >>>>>>>>>>> directory not in "/home/*". More about this can be found here [2]. >>>>>>>>>>> >>>>>>>>>>> Once the virtual machine is captured, it is stored in the >>>>>>>>>>> storage account that is associated with the virtual machine. You >>>>>>>>>>> can either >>>>>>>>>>> download this or use directly by referring to the URI when you want >>>>>>>>>>> to make >>>>>>>>>>> other virtual machines with this template. What would be awesome is >>>>>>>>>>> if we >>>>>>>>>>> can fully configure the virtual machine with a given product and >>>>>>>>>>> make it >>>>>>>>>>> available to users. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> Yes, this is mandatory. Otherwise we would not be able to >>>>>>>>>> autoscale a server cluster. >>>>>>>>>> >>>>>>>>>> I'm sorry I may have missed, how did it go with centralized >>>>>>>>>> logging? >>>>>>>>>> >>>>>>>>>> [3] >>>>>>>>>> http://docs.aws.amazon.com/autoscaling/latest/userguide/AutoScalingGroup.html >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> [1] >>>>>>>>>>> https://azure.microsoft.com/en-us/documentation/articles/load-balancer-overview/ >>>>>>>>>>> >>>>>>>>>>> [2] >>>>>>>>>>> https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-capture-image/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> On Wed, Jul 6, 2016 at 12:18 PM, Osura Rathnayake < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Akila, >>>>>>>>>>>> >>>>>>>>>>>> Please refer to the screenshots that I have attached. When I >>>>>>>>>>>> updated localMemberPort to 4200, I can see it being reflected in >>>>>>>>>>>> logs when >>>>>>>>>>>> members are joining. So should I still make modifications in the >>>>>>>>>>>> code? >>>>>>>>>>>> >>>>>>>>>>>> .gitignore was added. >>>>>>>>>>>> >>>>>>>>>>>> okay I will write test cases in testNG and update >>>>>>>>>>>> >>>>>>>>>>>> thanks, >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Jul 6, 2016 at 9:06 AM, Akila Ravihansa Perera < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Jul 5, 2016 at 11:38 AM, Osura Rathnayake < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Akila, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please check the modified code. It now takes the value which >>>>>>>>>>>>>> is specified as localMemberPort in axis2.xml. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I still don't see any change to the logic of how member >>>>>>>>>>>>> address is calculated. Can you double check? >>>>>>>>>>>>> >>>>>>>>>>>>> Please make sure "target/**" directories are ignored from >>>>>>>>>>>>> .gitignore. These shouldn't be in the repo [1]. You might also >>>>>>>>>>>>> need to >>>>>>>>>>>>> ignore any IDE specific files. Have a look at .gitignore in >>>>>>>>>>>>> Kubernetes >>>>>>>>>>>>> artifacts [2]. >>>>>>>>>>>>> >>>>>>>>>>>>> I see that you have committed some test cases based on JUnit. >>>>>>>>>>>>> Please note that as a platform we are moving to testng framework >>>>>>>>>>>>> so better >>>>>>>>>>>>> to use that. >>>>>>>>>>>>> @Imesh, Isuru: Please correct me if I'm wrong. >>>>>>>>>>>>> >>>>>>>>>>>>> Shall we get a repo created under wso2-incubator for this? >>>>>>>>>>>>> >>>>>>>>>>>>> [1] >>>>>>>>>>>>> https://github.com/osuran/azure-membership-scheme/tree/master/target >>>>>>>>>>>>> [2] >>>>>>>>>>>>> https://github.com/wso2/kubernetes-artifacts/blob/master/.gitignore >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks. >>>>>>>>>>>>> -- >>>>>>>>>>>>> Akila Ravihansa Perera >>>>>>>>>>>>> WSO2 Inc.; http://wso2.com/ >>>>>>>>>>>>> >>>>>>>>>>>>> Blog: http://ravihansa3000.blogspot.com >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Regards, >>>>>>>>>>>> Osura Rathnayake >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Regards, >>>>>>>>>>> Osura Rathnayake >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Imesh Gunaratne* >>>>>>>>>> Software Architect >>>>>>>>>> WSO2 Inc: http://wso2.com >>>>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057 >>>>>>>>>> W: https://medium.com/@imesh TW: @imesh >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> Osura Rathnayake >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Imesh Gunaratne* >>>>>>>> Software Architect >>>>>>>> WSO2 Inc: http://wso2.com >>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057 >>>>>>>> W: https://medium.com/@imesh TW: @imesh >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> Osura Rathnayake >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Thanks and Regards, >>>>>> >>>>>> Isuru H. >>>>>> +94 716 358 048* <http://wso2.com/>* >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> Osura Rathnayake >>>>> >>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Osura Rathnayake >>>> >>> >>> >>> >>> -- >>> *Imesh Gunaratne* >>> Software Architect >>> WSO2 Inc: http://wso2.com >>> T: +94 11 214 5345 M: +94 77 374 2057 >>> W: https://medium.com/@imesh TW: @imesh >>> >>> >> >> >> -- >> Thanks and Regards, >> >> Isuru H. >> +94 716 358 048* <http://wso2.com/>* >> >> >> > > > -- > Regards, > Osura Rathnayake > -- Thanks and Regards, Isuru H. +94 716 358 048* <http://wso2.com/>*
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
