Ah, I've just realised that the agent hasn't started in the cartridge. On Thu, May 29, 2014 at 7:45 AM, chris snow <[email protected]> wrote: > Thanks Nirmal. > > Non terminated member count doesn't seem to detect the two active > instances that have been started: > > [2014-05-29 06:06:23,900] DEBUG > {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum rule: > [network-partition] G1 [partition] P1 > [2014-05-29 06:06:23,906] DEBUG > {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check] > [network-partition] G1 [partition] P1 Non terminated member count: 0 > [2014-05-29 06:06:23,908] DEBUG > {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check] > [network-partition] G1 [partition] P1 Minimum member count: 1 > [2014-05-29 06:06:23,925] DEBUG > {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete > instances rule: [partition] P1 [network-partition] G1 > [2014-05-29 06:06:23,927] DEBUG > {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check] > [network-partition] G1 [partition] P1 Obsolete member count: 0 > [2014-05-29 06:06:23,930] INFO > {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient} > - Trying to spawn an instance via cloud controller: [cluster] > php111.php.domain [partition] P1 [lb-cluster] null > [network-partition-id] G1 > > > vagrant@puppet:~$ nova list > +--------------------------------------+---------------+--------+------------+-------------+--------------------+ > | ID | Name | Status | Task > State | Power State | Networks | > +--------------------------------------+---------------+--------+------------+-------------+--------------------+ > | a3ce6f52-27bf-4343-a05d-d7c9da9a1cb2 | php111php-3cb | ERROR | - > | NOSTATE | | > | 82434b60-6c9d-40a3-8191-2d466cde725e | php111php-47e | ERROR | - > | NOSTATE | | > | 4aa942ff-a722-4ff4-b604-abaa2f44c1a7 | php111php-499 | ERROR | - > | NOSTATE | | > | 1548915b-98f6-48fc-b70a-b0acc74610a7 | php111php-5e0 | ERROR | - > | NOSTATE | | > | f313c93d-7de1-4ccb-8c0d-f4104e5d837f | php111php-701 | ERROR | - > | NOSTATE | | > | a49598bb-0d4a-4064-af16-5b4b91057fd8 | php111php-719 | ERROR | - > | NOSTATE | | > | f8376091-99f6-4d17-8ec6-3935291c3e40 | php111php-841 | ERROR | - > | NOSTATE | | > | cf3298da-eecc-448a-bcea-2296eeed23dc | php111php-ad | ACTIVE | - > | Running | private=10.11.12.2 | > | 1d8832d5-39e4-476c-9c2b-1b1f5ffe3b41 | php111php-d81 | ACTIVE | - > | Running | private=10.11.12.3 | > | 4f6d51fb-fd27-4be1-94be-e9b5200fb6ba | php111php-e9c | ERROR | - > | NOSTATE | | > +--------------------------------------+---------------+--------+------------+-------------+--------------------+ > > I'll investigate some more... > > On Thu, May 29, 2014 at 7:42 AM, Nirmal Fernando <[email protected]> > wrote: >> Ah.. probably Chris. So, in min check what happens is it always make sure >> that you have min instance count in the system. >> >> >> On Thu, May 29, 2014 at 11:42 AM, chris snow <[email protected]> wrote: >>> >>> Hi Nirmal, >>> >>> I've just see that it is the "Minimal Rule" in mincheck.drl that is >>> trying to spawn the instance, and not scaling.drl. >>> >>> This is why AutoscaleAlgorithm.getNextScaleUpPartition() is not getting >>> called? >>> >>> Cheers, >>> >>> Chris >>> >>> >>> On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando >>> <[email protected]> wrote: >>> > No, Chris. This happens via Drools rules located in scaling.drl file of >>> > conf >>> > directory. >>> > >>> > >>> > On Thu, May 29, 2014 at 3:39 AM, chris snow <[email protected]> wrote: >>> >> >>> >> Hi Nirmal, >>> >> >>> >> For some reason, that method is not getting called. I've pasted my >>> >> stacktrace again with the drools methods removed: >>> >> >>> >> --- >>> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in >>> >> CloudControllerClient)) >>> >> owns: CloudControllerClient (id=2433) >>> >> owns: DefaultAgenda (id=2434) >>> >> CloudControllerClient.spawnAnInstance(Partition, String, String, >>> >> String) line: 137 >>> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String, String) >>> >> line: 92 >>> >> // >>> >> // drools methods hidden >>> >> // >>> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession, >>> >> FactHandle, Object) line: 86 >>> >> ClusterMonitor.monitor() line: 99 >>> >> ClusterMonitor.run() line: 72 >>> >> Thread.run() line: 744 >>> >> --- >>> >> >>> >> Would you expect to see AutoscaleAlgorithm.getNextScaleUpPartition >>> >> executed somewhere in the stack of calls? >>> >> >>> >> Many thanks, >>> >> >>> >> Chris >>> >> >>> >> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando >>> >> <[email protected]> wrote: >>> >> > Hi Chris, >>> >> > >>> >> > PartitionMax is considered in >>> >> > AutoscaleAlgorithm.getNextScaleUpPartition >>> >> > method. >>> >> > >>> >> > >>> >> > On Thu, May 29, 2014 at 3:09 AM, chris snow <[email protected]> >>> >> > wrote: >>> >> >> >>> >> >> I setup a breakpoint in eclipse at >>> >> >> CloudControllerClient.spawnAnInstance() and dumped the output >>> >> >> (below). >>> >> >> >>> >> >> I've scanned through the code in the stratos methods in the >>> >> >> stacktrace, but couldn't see any code that checks the partitionMax >>> >> >> value. >>> >> >> >>> >> >> Any ideas? >>> >> >> >>> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in >>> >> >> CloudControllerClient)) >>> >> >> owns: CloudControllerClient (id=2433) >>> >> >> owns: DefaultAgenda (id=2434) >>> >> >> CloudControllerClient.spawnAnInstance(Partition, String, String, >>> >> >> String) line: 137 >>> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String, >>> >> >> String) >>> >> >> line: 92 >>> >> >> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: >>> >> >> not available [native method] >>> >> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57 >>> >> >> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43 >>> >> >> Method.invoke(Object, Object...) line: 606 >>> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String, Object[], >>> >> >> Class[], ExecutableStatement[]) line: 1104 >>> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String) line: 987 >>> >> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377 >>> >> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext, >>> >> >> char[], >>> >> >> int, int, Object, Object, VariableResolverFactory, boolean, Class) >>> >> >> line: 143 >>> >> >> ASTNode.optimize(Object, Object, VariableResolverFactory) line: >>> >> >> 159 >>> >> >> ASTNode.getReducedValueAccelerated(Object, Object, >>> >> >> VariableResolverFactory) line: 115 >>> >> >> MVELRuntime.execute(boolean, CompiledExpression, Object, >>> >> >> VariableResolverFactory) line: 85 >>> >> >> CompiledExpression.getDirectValue(Object, >>> >> >> VariableResolverFactory) >>> >> >> line: 123 >>> >> >> CompiledExpression.getValue(Object, VariableResolverFactory) >>> >> >> line: >>> >> >> 119 >>> >> >> MVEL.executeExpression(Object, Object, VariableResolverFactory) >>> >> >> line: >>> >> >> 930 >>> >> >> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory) line: >>> >> >> 104 >>> >> >> DefaultAgenda.fireActivation(Activation) line: 1287 >>> >> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221 >>> >> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456 >>> >> >> >>> >> >> >>> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter, >>> >> >> int) line: 710 >>> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules() line: >>> >> >> 674 >>> >> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230 >>> >> >> >>> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession, >>> >> >> FactHandle, Object) line: 86 >>> >> >> ClusterMonitor.monitor() line: 99 >>> >> >> ClusterMonitor.run() line: 72 >>> >> >> Thread.run() line: 744 >>> >> >> >>> >> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan >>> >> >> <[email protected]> >>> >> >> wrote: >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow <[email protected]> >>> >> >> > wrote: >>> >> >> >> >>> >> >> >> Hi Lahiru, >>> >> >> >> >>> >> >> >> I'll have a poke around - it will give me a good opportunity to >>> >> >> >> learn >>> >> >> >> some more Stratos code :) >>> >> >> >> >>> >> >> > >>> >> >> > Cool :) >>> >> >> > >>> >> >> > Debug logs should be able to show us where it went wrong. >>> >> >> > >>> >> >> > Thanks. >>> >> >> >> >>> >> >> >> Many thanks, >>> >> >> >> >>> >> >> >> Chris >>> >> >> >> >>> >> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan >>> >> >> >> <[email protected]> >>> >> >> >> wrote: >>> >> >> >> > Hi Chris, >>> >> >> >> > >>> >> >> >> > Can you enable debug logs and try out? >>> >> >> >> > If so please send them over. >>> >> >> >> > >>> >> >> >> > Thanks. >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow >>> >> >> >> > <[email protected]> >>> >> >> >> > wrote: >>> >> >> >> >> >>> >> >> >> >> Deployment Policy partitionMax seems to be ignored. My >>> >> >> >> >> deployment >>> >> >> >> >> policy looks like this: >>> >> >> >> >> >>> >> >> >> >> { >>> >> >> >> >> "id": "economyDeployment", >>> >> >> >> >> "partitionGroup": [ >>> >> >> >> >> { >>> >> >> >> >> "id": "G1", >>> >> >> >> >> "partitionAlgo": "one-after-another", >>> >> >> >> >> "partition": [ >>> >> >> >> >> { >>> >> >> >> >> "id": "P1", >>> >> >> >> >> "partitionMax": "1", >>> >> >> >> >> "partitionMin": "1" >>> >> >> >> >> } >>> >> >> >> >> ] >>> >> >> >> >> } >>> >> >> >> >> ] >>> >> >> >> >> } >>> >> >> >> >> >>> >> >> >> >> Stratos initially starts two instances: >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+ >>> >> >> >> >> | ID | Name | >>> >> >> >> >> Status | >>> >> >> >> >> Task >>> >> >> >> >> State | Power State | Networks | >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+ >>> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | >>> >> >> >> >> ACTIVE | >>> >> >> >> >> - >>> >> >> >> >> | Running | private=10.11.12.2 | >>> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | >>> >> >> >> >> ACTIVE | >>> >> >> >> >> - >>> >> >> >> >> | Running | private=10.11.12.3 | >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+ >>> >> >> >> >> Then attempts to start more: >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+ >>> >> >> >> >> | ID | Name | >>> >> >> >> >> Status | >>> >> >> >> >> Task >>> >> >> >> >> State | Power State | Networks | >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+ >>> >> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR >>> >> >> >> >> | >>> >> >> >> >> - >>> >> >> >> >> | NOSTATE | | >>> >> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR >>> >> >> >> >> | >>> >> >> >> >> - >>> >> >> >> >> | NOSTATE | | >>> >> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR >>> >> >> >> >> | >>> >> >> >> >> - >>> >> >> >> >> | NOSTATE | | >>> >> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR >>> >> >> >> >> | >>> >> >> >> >> - >>> >> >> >> >> | NOSTATE | | >>> >> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR >>> >> >> >> >> | >>> >> >> >> >> - >>> >> >> >> >> | NOSTATE | | >>> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | >>> >> >> >> >> ACTIVE | >>> >> >> >> >> - >>> >> >> >> >> | Running | private=10.11.12.2 | >>> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | >>> >> >> >> >> ACTIVE | >>> >> >> >> >> - >>> >> >> >> >> | Running | private=10.11.12.3 | >>> >> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR >>> >> >> >> >> | >>> >> >> >> >> - >>> >> >> >> >> | NOSTATE | | >>> >> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR >>> >> >> >> >> | >>> >> >> >> >> - >>> >> >> >> >> | NOSTATE | | >>> >> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR >>> >> >> >> >> | >>> >> >> >> >> - >>> >> >> >> >> | NOSTATE | | >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+ >>> >> >> >> >> >>> >> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana + >>> >> >> >> >> Docker. >>> >> >> >> >> >>> >> >> >> >> Any ideas? >>> >> >> >> >> >>> >> >> >> >> Many thanks, >>> >> >> >> >> >>> >> >> >> >> Chris >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > -- >>> >> >> >> > -- >>> >> >> >> > Lahiru Sandaruwan >>> >> >> >> > Committer and PMC member, Apache Stratos, >>> >> >> >> > Senior Software Engineer, >>> >> >> >> > WSO2 Inc., http://wso2.com >>> >> >> >> > lean.enterprise.middleware >>> >> >> >> > >>> >> >> >> > email: [email protected] cell: (+94) 773 325 954 >>> >> >> >> > blog: http://lahiruwrites.blogspot.com/ >>> >> >> >> > twitter: http://twitter.com/lahirus >>> >> >> >> > linked-in: >>> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>> >> >> >> > >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> -- >>> >> >> >> Check out my professional profile and connect with me on >>> >> >> >> LinkedIn. >>> >> >> >> http://lnkd.in/cw5k69 >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > -- >>> >> >> > -- >>> >> >> > Lahiru Sandaruwan >>> >> >> > Committer and PMC member, Apache Stratos, >>> >> >> > Senior Software Engineer, >>> >> >> > WSO2 Inc., http://wso2.com >>> >> >> > lean.enterprise.middleware >>> >> >> > >>> >> >> > email: [email protected] cell: (+94) 773 325 954 >>> >> >> > blog: http://lahiruwrites.blogspot.com/ >>> >> >> > twitter: http://twitter.com/lahirus >>> >> >> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>> >> >> > >>> >> >> >>> >> >> >>> >> >> >>> >> >> -- >>> >> >> Check out my professional profile and connect with me on LinkedIn. >>> >> >> http://lnkd.in/cw5k69 >>> >> > >>> >> > >>> >> > >>> >> > >>> >> > -- >>> >> > Best Regards, >>> >> > Nirmal >>> >> > >>> >> > Nirmal Fernando. >>> >> > PPMC Member & Committer of Apache Stratos, >>> >> > >>> >> > Senior Software Engineer, WSO2 Inc. >>> >> > >>> >> > Blog: http://nirmalfdo.blogspot.com/ >>> >> >>> >> >>> >> >>> >> -- >>> >> Check out my professional profile and connect with me on LinkedIn. >>> >> http://lnkd.in/cw5k69 >>> > >>> > >>> > >>> > >>> > -- >>> > Best Regards, >>> > Nirmal >>> > >>> > Nirmal Fernando. >>> > PPMC Member & Committer of Apache Stratos, >>> > Senior Software Engineer, WSO2 Inc. >>> > >>> > Blog: http://nirmalfdo.blogspot.com/ >>> >>> >>> >>> -- >>> Check out my professional profile and connect with me on LinkedIn. >>> http://lnkd.in/cw5k69 >> >> >> >> >> -- >> Best Regards, >> Nirmal >> >> Nirmal Fernando. >> PPMC Member & Committer of Apache Stratos, >> Senior Software Engineer, WSO2 Inc. >> >> Blog: http://nirmalfdo.blogspot.com/ > > > > -- > Check out my professional profile and connect with me on LinkedIn. > http://lnkd.in/cw5k69
-- Check out my professional profile and connect with me on LinkedIn. http://lnkd.in/cw5k69
