On Mon, May 18, 2015 at 1:37 AM, Lahiru Sandaruwan <[email protected]> wrote:
> Hi, > > I guess the issue is in "handleAverageRequestsInFlightEvent" method of > ClusterMonitor class. The difference in this case, is the cluster instance > id is "not-defined", and in mock iaas case, it send the cluster instance id. > > I found an issue at setting LB stats for instance context in above case and just fixed it. Thanks. > Thanks. > > On Mon, May 18, 2015 at 1:10 AM, Imesh Gunaratne <[email protected]> wrote: > >> Great work Lahiru! I just retested it with Kubernetes including your fix. >> But still it's not scaling up for me. Please see the attached logs. >> >> Thanks >> >> On Mon, May 18, 2015 at 12:56 AM, Lahiru Sandaruwan <[email protected]> >> wrote: >> >>> Hi, >>> >>> I was able to run a test successfully with mock IaaS now. >>> See the following logs, >>> >>> [2015-05-18 00:53:27,794] DEBUG >>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] [cluster] >>> single-cartridge-app.my-php.php.domain RIF predicted value: 100.0 >>> [2015-05-18 00:53:27,794] DEBUG >>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] [cluster] >>> single-cartridge-app.my-php.php.domain RIF threshold: 50.0 >>> [2015-05-18 00:53:27,795] DEBUG >>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] [cluster] >>> single-cartridge-app.my-php.php.domain MC predicted value: 20.0 >>> [2015-05-18 00:53:27,795] DEBUG >>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] [cluster] >>> single-cartridge-app.my-php.php.domain MC threshold: 40.0 >>> [2015-05-18 00:53:27,795] DEBUG >>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] [cluster] >>> single-cartridge-app.my-php.php.domain LA predicted value: 20.0 >>> [2015-05-18 00:53:27,796] DEBUG >>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] [cluster] >>> single-cartridge-app.my-php.php.domain LA threshold: 40.0 >>> [2015-05-18 00:53:27,828] DEBUG >>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] [cluster] >>> single-cartridge-app.my-php.php.domain Scale-up action: false >>> [2015-05-18 00:53:27,828] DEBUG >>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] [cluster] >>> single-cartridge-app.my-php.php.domain Scale-down action: false >>> [2015-05-18 00:53:27,830] DEBUG >>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] Number of >>> required instances based on stats: 2 [active instances count] 2 >>> [network-partition] network-partition-1 [cluster] >>> single-cartridge-app.my-php.php.domain >>> >>> >>> Thanks. >>> >>> On Sun, May 17, 2015 at 10:43 PM, Imesh Gunaratne <[email protected]> >>> wrote: >>> >>>> There were few issues in the CEP artifacts and fixed them. Lahiru is >>>> now working on fixing another issue in RuleTaskDelegator. >>>> >>>> Thanks >>>> >>>> On Sun, May 17, 2015 at 9:20 PM, Lahiru Sandaruwan <[email protected]> >>>> wrote: >>>> >>>>> It seems the gradients are negative. This would happen if there has >>>>> been some values and then reduced... >>>>> >>>>> I will also investigate. >>>>> >>>>> Thanks. >>>>> >>>>> On Sun, May 17, 2015 at 1:43 PM, Imesh Gunaratne <[email protected]> >>>>> wrote: >>>>> >>>>>> It looks like RIF values are not received by the Autoscaler, need to >>>>>> investigate. >>>>>> >>>>>> On Sun, May 17, 2015 at 12:59 PM, Imesh Gunaratne <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Lahiru, >>>>>>> >>>>>>> The above issue was fixed, now I'm seeing a problem in Autoscaler >>>>>>> where it calculates the RIF gradients as negative values: >>>>>>> >>>>>>> *Scenario:* >>>>>>> *Autoscaling policy:* requestsInFlight/threshold -> 50 >>>>>>> *Incoming request load:* Concurrency -> 80, number of requests >>>>>>> ->1000 >>>>>>> *RIF published by load balancer:* 80 >>>>>>> >>>>>>> [2015-05-17 12:50:39,290] DEBUG >>>>>>> {org.apache.stratos.autoscaler.monitor.cluster.ClusterMonitor} - >>>>>>> Second derivative of Rif event: [cluster] >>>>>>> single-cartridge-app.my-php.php.domain [network-partition] >>>>>>> network-partition-1 [value] -0.1776534 >>>>>>> [2015-05-17 12:50:39,291] DEBUG >>>>>>> {org.apache.stratos.autoscaler.monitor.cluster.ClusterMonitor} >>>>>>> - Gradient of Rif event: [cluster] >>>>>>> single-cartridge-app.my-php.php.domain >>>>>>> [network-partition] network-partition-1 [value] -1.7770275 >>>>>>> >>>>>>> May be due to this reason scaling up decision was not taken even >>>>>>> though the RIF values are greater than the defined threshold. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> On Sun, May 17, 2015 at 12:12 PM, Imesh Gunaratne <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Devs, >>>>>>>> >>>>>>>> While working on $subject and I found that we have not packaged the >>>>>>>> thrift client configuration file in the load balancer distribution: >>>>>>>> >>>>>>>> [2015-05-17 12:05:07,746] FATAL - LoadBalancerServiceComponent >>>>>>>> Failed to activate load balancer service component >>>>>>>> java.lang.RuntimeException: Could not parse thrift client >>>>>>>> configuration >>>>>>>> at >>>>>>>> org.apache.stratos.common.statistics.publisher.wso2.cep.ThriftClientConfigParser.parse(ThriftClientConfigParser.java:136) >>>>>>>> at >>>>>>>> org.apache.stratos.common.statistics.publisher.wso2.cep.ThriftClientConfig.getInstance(ThriftClientConfig.java:45) >>>>>>>> at >>>>>>>> org.apache.stratos.common.statistics.publisher.wso2.cep.WSO2CEPStatisticsPublisher.<init>(WSO2CEPStatisticsPublisher.java:56) >>>>>>>> >>>>>>>> I'm now fixing this issue and continue the test. >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Imesh Gunaratne >>>>>>>> >>>>>>>> Senior Technical Lead, WSO2 >>>>>>>> Committer & PMC Member, Apache Stratos >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Imesh Gunaratne >>>>>>> >>>>>>> Senior Technical Lead, WSO2 >>>>>>> Committer & PMC Member, Apache Stratos >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Imesh Gunaratne >>>>>> >>>>>> Senior Technical Lead, WSO2 >>>>>> Committer & PMC Member, Apache Stratos >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> -- >>>>> Lahiru Sandaruwan >>>>> Committer and PMC member, Apache Stratos, >>>>> Senior Software Engineer, >>>>> WSO2 Inc., http://wso2.com >>>>> lean.enterprise.middleware >>>>> >>>>> phone: +94773325954 >>>>> email: [email protected] blog: http://lahiruwrites.blogspot.com/ >>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>>>> >>>>> >>>> >>>> >>>> -- >>>> Imesh Gunaratne >>>> >>>> Senior Technical Lead, WSO2 >>>> Committer & PMC Member, Apache Stratos >>>> >>> >>> >>> >>> -- >>> -- >>> Lahiru Sandaruwan >>> Committer and PMC member, Apache Stratos, >>> Senior Software Engineer, >>> WSO2 Inc., http://wso2.com >>> lean.enterprise.middleware >>> >>> phone: +94773325954 >>> email: [email protected] blog: http://lahiruwrites.blogspot.com/ >>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>> >>> >> >> >> -- >> Imesh Gunaratne >> >> Senior Technical Lead, WSO2 >> Committer & PMC Member, Apache Stratos >> > > > > -- > -- > Lahiru Sandaruwan > Committer and PMC member, Apache Stratos, > Senior Software Engineer, > WSO2 Inc., http://wso2.com > lean.enterprise.middleware > > phone: +94773325954 > email: [email protected] blog: http://lahiruwrites.blogspot.com/ > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 > > -- -- Lahiru Sandaruwan Committer and PMC member, Apache Stratos, Senior Software Engineer, WSO2 Inc., http://wso2.com lean.enterprise.middleware phone: +94773325954 email: [email protected] blog: http://lahiruwrites.blogspot.com/ linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
