In other words, fix reduces the risks which are coming from bad samples.
On Thu, Nov 6, 2014 at 12:08 PM, Rajkumar Rajaratnam <rajkum...@wso2.com>
wrote:
>
>
> On Thu, Nov 6, 2014 at 11:38 AM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam <rajkum...@wso2.com>
>> wrote:
>>
>>> Hi,
>>>
>>> No Imesh. There are no difference between *taking the the difference
>>> and dividing it by 1000* and *dividing it by 1000 and taking the
>>> difference*.
>>>
>>
>> Raj: I agree, in this scenario it makes no difference, however when we
>> are applying a fomular the unit conversion should happen prior to applying
>> values.
>>
>
> Not exactly Imesh. We can do operations if operands are in the same units.
> So we can do subtract operation in milliseconds. And then we can do
> division operation to make it in seconds. On the other hand, we can do a
> division operation to make each operands to seconds, then since both are
> seconds, we can do a subtract operation. So we can apply the formula if
> operands are in same unit. I guess this is same for all scenarios.
>
>
>> Still I see a problem with this correction, take the same example:
>> Gradient: -999.9999999999998 Last val: 7.000000000000001 First val: 12.0
>> Time Gap: 5 t1: 1415213232152 t2: 1415213232157
>>
>> *Manual calulation:*
>> The value difference: 7 - 12 = -5
>> Time difference: 1415213232157 - 1415213232152 = 5 ms => 5/1000 = 0.005
>> sec
>> *Gradient = - 5/0.005 = -1000*
>>
>> *According to the current fix:*
>> long millisecondsForASecond = 1000;
>> long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 :
>> millisecondsForASecond;
>> double gradient = 0.0;
>> if (tGap > 0) {
>> gradient = ((lastVal - firstVal) * millisecondsForASecond) /
>> tGap;
>> }
>>
>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>> Gradient = -5 * 1000 / 1000 = - 5
>>
>
> IMO, this is a sufficient solution.
>
> Let me explain how I see Nirmal's solution.
>
> We are taking only first and last values in a window for gradient
> calculation. If there are more events in that window, and if that is a good
> sample, tGap will be taking some higher values than 1s. This is usual
> situation. Nirmal's fix is nothing to do with this scenario.
>
> If there are few events in that window(say only 2), they can be really
> nearer. So there is a possibility that tGap can take a value near to 0. In
> that case gradient will be a very big value. Hence, if the tGap is very low
> (less than 1 second), then we can assume that the first and last events
> actually occurred in 1 second gap. The reason why tGap is less than 0 is,
> there are only few events in that window, and that sample is not good
> enough. But this a rare situation, it can only happen once in a while.
> Nirmal's fix is only dealing with this scenario. Fix imposes a lower limit
> in the time interval between two events. This fix is really important to
> deal with samples which are not good enough.
>
> By previous formula, we get -1000. This happened because the sample is not
> good enough (may be there are few events). Fix makes it to -5.
>
> So my understanding is that the solution is good enough for now.
>
> Thanks.
>
>>
>> Nirmal: Can you please justify this?
>>
>> Thanks
>>
>> On Thu, Nov 6, 2014 at 11:31 AM, Nirmal Fernando <nirmal070...@gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Thu, Nov 6, 2014 at 6:05 AM, Gayan Gunarathne <gay...@wso2.com>
>>> wrote:
>>>
>>>> AFAIU this is not related with the gradient calculation formula that we
>>>> used. But It is related with the how we calculate gradient when the
>>>> denominator become a small number.IMO when the divisor (timeInSeconds)
>>>> approaches to the small number we do need to normalize the result.
>>>>
>>>> IMO we do have max and min scale up and down values. So based on that
>>>> we can calculate the gradient on that value range.Simply if the denominator
>>>> approaches zero result approaches to infinity.
>>>>
>>>> I think that's what Nirmal doing here by making the lower threshold as
>>>> "millisecondsForASecond"
>>>>
>>>
>>> You got it! thanks.
>>>
>>>>
>>>>
>>>> long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 :
>>>> millisecondsForASecond;
>>>> double gradient = 0.0;
>>>> if (tGap > 0) {
>>>> gradient = ((lastVal - firstVal) * millisecondsForASecond)
>>>> / tGap;
>>>> }
>>>>
>>>>
>>>> Thanks,
>>>> Gayan
>>>>
>>>> On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam <rajkum...@wso2.com
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> On Thu, Nov 6, 2014 at 9:03 AM, Imesh Gunaratne <im...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi Nirmal,
>>>>>>
>>>>>> I do not think the milliseconds to seconds convertion is correct
>>>>>> here.
>>>>>>
>>>>>> As I see we are taking the difference of two timestamp values and
>>>>>> then dividing it by 1000. The corect way might be to first divide each
>>>>>> value by 1000 and then take the difference.
>>>>>>
>>>>>
>>>>> No Imesh. There are no difference between *taking the the difference
>>>>> and dividing it by 1000* and *dividing it by 1000 and taking the
>>>>> difference*.
>>>>>
>>>>>
>>>>>> On the other hand we might not need to convert these values to
>>>>>> seconds since we are taking a time difference and calculating a gradient.
>>>>>>
>>>>>> I did a quick test with the following sample:
>>>>>> Gradient: -999.9999999999998 Last val: 7.000000000000001 First val:
>>>>>> 12.0 Time Gap: 5 t1: 1415213232152 t2: 1415213232157
>>>>>>
>>>>>> *According previous code:*
>>>>>> long tGap = t2 - t1;
>>>>>> double gradient = 0.0;
>>>>>> if (tGap > 0) {
>>>>>> gradient = ((lastVal - firstVal) * 1000) / tGap;
>>>>>> }
>>>>>>
>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>> gradient: -1000.0
>>>>>>
>>>>>> *According to your fix:*
>>>>>> long millisecondsForASecond = 1000;
>>>>>> long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 :
>>>>>> millisecondsForASecond;
>>>>>> double gradient = 0.0;
>>>>>> if (tGap > 0) {
>>>>>> gradient = ((lastVal - firstVal) *
>>>>>> millisecondsForASecond) / tGap;
>>>>>> }
>>>>>>
>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>> gradient: -5.0
>>>>>>
>>>>>> *According to an online gradient calculator:*
>>>>>> gradient: -1
>>>>>>
>>>>>> http://www.calculator.net/slope-calculator.html?type=1&x11=1415213232152&y11=12&x12=1415213232157&y12=7&x=27&y=19
>>>>>>
>>>>>> According to the online gradient calculator (assuming their
>>>>>> calculation is correct), the calculation in your fix is not correct. I
>>>>>> believe the logic should be simple as follows:
>>>>>>
>>>>>> long tGap = t2 - t1;
>>>>>> double gradient = 0.0;
>>>>>> if (tGap > 0) {
>>>>>> gradient = ((lastVal - firstVal)) / tGap;
>>>>>> }
>>>>>>
>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7
>>>>>> gradient: -1.0
>>>>>>
>>>>>
>>>>> And AFAIK, standard way to calculate gradient = (v1-v2)/timeInSeconds.
>>>>>
>>>>> Well, we can divide it by what ever time units we wants, but the
>>>>> result's unit will be different.
>>>>>
>>>>> In Nirmal's case, he get will get *stats/seconds*. In your case, you
>>>>> will get *stats/milliseconds*
>>>>>
>>>>> And AS prediction is based on *seconds*, CEP should use *seconds* for
>>>>> the calculation.
>>>>>
>>>>> wdyt?
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>> On Thu, Nov 6, 2014 at 1:32 AM, Nirmal Fernando <
>>>>>> nirmal070...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Guys,
>>>>>>>
>>>>>>> So, I got a chance to reproduce and analyze this issue.
>>>>>>>
>>>>>>> How we calculate gradient of two events?
>>>>>>>
>>>>>>> Say the events are; e1(t1,v1) and e2(t2,v2)
>>>>>>>
>>>>>>> tx - time in *milliseconds* when xth event occurred
>>>>>>> vx - value (memory, cpu etc.) that xth event carries
>>>>>>>
>>>>>>> *time gap = t(2-1) = t2 -t1 milliseconds*
>>>>>>> time gap in seconds = t(2-1) = (t2 - t1)/1000 seconds
>>>>>>>
>>>>>>> Hence,
>>>>>>> *Gradient = (v2 - v1) / t(2-1) = ( (v2 - v1) * 1000 ) / (t2 - t1)*
>>>>>>>
>>>>>>> I've enabled debug logs for CEP extension;
>>>>>>>
>>>>>>> log4j.logger.org.apache.stratos.cep.extension=DEBUG
>>>>>>>
>>>>>>> Please find the following 3 logs extracted from the debug logs;
>>>>>>>
>>>>>>> ===================================================================
>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG
>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor}
>>>>>>> -
>>>>>>> Gradient: -0.1996007984031936 Last val: 9.0 First val: 12.0 *Time
>>>>>>> Gap: 15030* t1: 1415213202095 t2: 1415213217125 hash: 155426542
>>>>>>>
>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG
>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor}
>>>>>>> - Gradient:
>>>>>>> -999.9999999999998 Last val: 7.000000000000001 First val: 12.0 *Time
>>>>>>> Gap: 5* t1: 1415213232152 t2: 1415213232157 hash: 155426542
>>>>>>>
>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,074] DEBUG
>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor}
>>>>>>> -
>>>>>>> Gradient: -44.34884666437174 Last val: -999.9999999999998 First val:
>>>>>>> -0.1996007984031936 *Time Gap: 22544* t1: 1415213209610 t2:
>>>>>>> 1415213232154 hash: 155426542
>>>>>>> ===================================================================
>>>>>>>
>>>>>>> So, as you can see the reason behind a large value is when the time
>>>>>>> gap between two subjected events is less than 1 second. This could
>>>>>>> happen
>>>>>>> since events are coming from different asynchronous agents and also when
>>>>>>> there are less number of events.
>>>>>>>
>>>>>>> So, the fix I propose is a very simple one and it will not
>>>>>>> compromise anything AFAIS.
>>>>>>>
>>>>>>> Fix is to calculate time gap as follows;
>>>>>>> *____ t2 -
>>>>>>> t1*
>>>>>>> *| yes?*
>>>>>>> *time gap = t(2-1) = t2 -t1 > 1000 ---*
>>>>>>> *|____ 1000*
>>>>>>>
>>>>>>> I have tested this and works fine.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---------- Forwarded message ----------
>>>>>>> From: Manula Chathurika Thantriwatte <manu...@wso2.com>
>>>>>>> Date: Tue, Oct 28, 2014 at 5:16 AM
>>>>>>> Subject: Re: CEP sends very large values for gradient and second
>>>>>>> derivative of load average
>>>>>>> To: "dev@stratos.apache.org" <dev@stratos.apache.org>
>>>>>>>
>>>>>>>
>>>>>>> Hi Raj,
>>>>>>>
>>>>>>> Yes it has. But it's very easy to find the values from the agent
>>>>>>> with the time stamp. Then we can search that time stamp in the CEP trace
>>>>>>> logs and compare the summarized values.
>>>>>>>
>>>>>>> Thanks !
>>>>>>>
>>>>>>> On Mon, Oct 27, 2014 at 9:23 PM, Rajkumar Rajaratnam <
>>>>>>> rajkum...@wso2.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Oct 27, 2014 at 7:56 PM, Manula Chathurika Thantriwatte <
>>>>>>>> manu...@wso2.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Raj,
>>>>>>>>>
>>>>>>>>> Shall we enable the cartridge agent debug logs. Then we can
>>>>>>>>> compare both CEP and agent logs with the time stamp. Then we can
>>>>>>>>> narrow
>>>>>>>>> down where the actual problem is.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Manula, cep-trace log have the values sent by cartridge agent also
>>>>>>>> right?
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks !
>>>>>>>>>
>>>>>>>>> On Mon, Oct 27, 2014 at 5:06 PM, Rajkumar Rajaratnam <
>>>>>>>>> rajkum...@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Oct 27, 2014 at 12:53 PM, Manula Chathurika Thantriwatte
>>>>>>>>>> <manu...@wso2.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>
>>>>>>>>>>> Is this happen frequently or intermittently ? Once I go though
>>>>>>>>>>> the logs I'm unable to find the high input values for load average.
>>>>>>>>>>> But
>>>>>>>>>>> output have high values.
>>>>>>>>>>>
>>>>>>>>>>> 22:10:38,698 [-] [Siddhi-Scheduler-pool-10-thread-1] INFO
>>>>>>>>>>> EVENT_TRACE_LOGGER TenantId=-1234 : Output Event Adaptor :
>>>>>>>>>>> JMSOutputAdaptor, sent
>>>>>>>>>>>
>>>>>>>>>>> {"org.apache.stratos.messaging.event.health.stat.SecondDerivativeOfLoadAverageEvent":{"message":{"clusterId":"php.php.domain","networkPartitionId":"","value":"2000000.0"}}}
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This is happening intermittently. Experienced now too. This is a
>>>>>>>>>> serious issue. We need to somehow sort this out.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Also networkPartitionId is empty. In the current execution plans
>>>>>>>>>>> it took networkPartitionId for the calculation. IMO we need to
>>>>>>>>>>> update the
>>>>>>>>>>> CEP execution plans for 4.1.0.
>>>>>>>>>>>
>>>>>>>>>>> Thanks !
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:49 AM, Manula Chathurika Thantriwatte
>>>>>>>>>>> <manu...@wso2.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks Raj. I'll go through them.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:33 AM, Rajkumar Rajaratnam <
>>>>>>>>>>>> rajkum...@wso2.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Manula,
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can see those values in the same log I attached in the
>>>>>>>>>>>>> previous mail.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:29 AM, Rajkumar Rajaratnam <
>>>>>>>>>>>>> rajkum...@wso2.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Chamila,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I couldn't find any large values sent by agent. There are
>>>>>>>>>>>>>> traces for CEP sending large values. Find the cep trace log here
>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1.
>>>>>>>>>>>>>> https://drive.google.com/file/d/0B1haIleqJMHIS3FFVFpfa0JlbVE/view?usp=sharing
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:23 AM, Manula Chathurika
>>>>>>>>>>>>>> Thantriwatte <manu...@wso2.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Shall we enable the cartridge agent debug logs and see what
>>>>>>>>>>>>>>> are the load average values sent from the agent. If those
>>>>>>>>>>>>>>> values are normal
>>>>>>>>>>>>>>> then we can narrow down to the CEP.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks !
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:43 PM, Chamila De Alwis <
>>>>>>>>>>>>>>> chami...@wso2.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Raj,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Can you inspect the CEP trace logs for the same time
>>>>>>>>>>>>>>>> period? It will have the events published from the agent.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Chamila de Alwis
>>>>>>>>>>>>>>>> Software Engineer | WSO2 | +94772207163
>>>>>>>>>>>>>>>> Blog: code.chamiladealwis.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:39 PM, Rajkumar Rajaratnam <
>>>>>>>>>>>>>>>> rajkum...@wso2.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I am doing a test round with M3 with python agent. And
>>>>>>>>>>>>>>>>> faced this very strange scenario, $Subject.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] -2000.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 1.9090909
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value]
>>>>>>>>>>>>>>>>> 28.718182
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Second
>>>>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster]
>>>>>>>>>>>>>>>>> php.php.domain [value]
>>>>>>>>>>>>>>>>> 200000.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Second
>>>>>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain
>>>>>>>>>>>>>>>>> [value] 0.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Second
>>>>>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain
>>>>>>>>>>>>>>>>> [value] 0.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 3.3636363
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value]
>>>>>>>>>>>>>>>>> 0.11670045
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value]
>>>>>>>>>>>>>>>>> 28.7
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Second
>>>>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster]
>>>>>>>>>>>>>>>>> php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value]
>>>>>>>>>>>>>>>>> 28.7
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 5.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 2000.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value]
>>>>>>>>>>>>>>>>> 28.7
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Second
>>>>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster]
>>>>>>>>>>>>>>>>> php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Second
>>>>>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain
>>>>>>>>>>>>>>>>> [value] 2000000.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 13500.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value]
>>>>>>>>>>>>>>>>> 50.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value]
>>>>>>>>>>>>>>>>> 28.738462
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 2.3333333
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 31.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 39.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 39.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,705] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] -11000.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Load average
>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Grad of
>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor}
>>>>>>>>>>>>>>>>> - Avg
>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value]
>>>>>>>>>>>>>>>>> 28.8
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} -
>>>>>>>>>>>>>>>>> Memory
>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub
>>>>>>>>>>>>>>>>> cluster] KubGrp1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,811] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher}
>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>> PodActivationWatcher running : Running
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,942] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher}
>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>> PodActivationWatcher running : Running
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,945] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher}
>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>> PodActivationWatcher running : Running
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,712] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor}
>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>> KubernetesServiceClusterMonitor is running..
>>>>>>>>>>>>>>>>> KubernetesServiceClusterMonitor [
>>>>>>>>>>>>>>>>> kubernetesHostClusterId=KubGrp1,
>>>>>>>>>>>>>>>>> clusterId=php.php.domain, serviceId=php]
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor}
>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>> Running min check for [kub-cluster] : KubGrp1 [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - Running
>>>>>>>>>>>>>>>>> minimum rule:
>>>>>>>>>>>>>>>>> [kub-cluster] KubGrp1 [cluster] php.php.domain
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - Running
>>>>>>>>>>>>>>>>> obsolete containers
>>>>>>>>>>>>>>>>> rule [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} -
>>>>>>>>>>>>>>>>> [obsolete-check] [cluster]
>>>>>>>>>>>>>>>>> : php.php.domain [Replicas] obsoleteReplicas : 0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator}
>>>>>>>>>>>>>>>>> - Minimum
>>>>>>>>>>>>>>>>> check executed for :
>>>>>>>>>>>>>>>>> org.apache.stratos.autoscaler.KubernetesClusterContext@21d906d0
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor}
>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>> flag of rifReset : true flag of memoryConsumptionReset :
>>>>>>>>>>>>>>>>> true flag of
>>>>>>>>>>>>>>>>> loadAverageReset : true
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor}
>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>> Running scale check for [kub-cluster] : KubGrp1 [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - Running
>>>>>>>>>>>>>>>>> scaling rule
>>>>>>>>>>>>>>>>> [kub-cluster] : KubGrp1 [cluster] : php.php.domain
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -
>>>>>>>>>>>>>>>>> Predicting the
>>>>>>>>>>>>>>>>> value, [average]: 0.0 , [gradient]: 0.0 , [second
>>>>>>>>>>>>>>>>> derivative]: 0.0 , [time
>>>>>>>>>>>>>>>>> intervals]: 1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -
>>>>>>>>>>>>>>>>> Predicting the
>>>>>>>>>>>>>>>>> value, [average]: 28.8 , [gradient]: 0.0 , [second
>>>>>>>>>>>>>>>>> derivative]: 0.0 , [time
>>>>>>>>>>>>>>>>> intervals]: 1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} -
>>>>>>>>>>>>>>>>> Predicting the
>>>>>>>>>>>>>>>>> value, [average]: 39.0 , [gradient]: -11000.0 , [second
>>>>>>>>>>>>>>>>> derivative]:
>>>>>>>>>>>>>>>>> 2000000.0 , [time intervals]: 1
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [Replicas] maxReplicas : 10
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [Replicas] activeReplicas : 3
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [RequestInFlight] predicted value : 0.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [RequestInFlight] upper limit : 80.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [RequestInFlight] lower limit : 5.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] predicted value :
>>>>>>>>>>>>>>>>> 28.799999237060547
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] upper limit : 80.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] lower limit : 15.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] predicted value : 989039.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] upper limit : 180.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] lower limit : 20.0
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain scale-up action : true
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain scale-down action : false
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] predicted replicas : 16484
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain predicted replicas > max replicas :
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling]
>>>>>>>>>>>>>>>>> Decided to
>>>>>>>>>>>>>>>>> scale-up : [cluster] : php.php.domain
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling-up]
>>>>>>>>>>>>>>>>> [cluster] :
>>>>>>>>>>>>>>>>> php.php.domain valid number of replicas to expand : 10
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO
>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
>>>>>>>>>>>>>>>>> - Updating kubernetes replication controller via cloud
>>>>>>>>>>>>>>>>> controller:
>>>>>>>>>>>>>>>>> [cluster] php.php.domain [replicas] 10
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,734] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl}
>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>> CloudControllerServiceImpl:updateContainers for cluster :
>>>>>>>>>>>>>>>>> php.php.domain
>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,758] DEBUG
>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl}
>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>> Cloud Controller is delegating request to update a
>>>>>>>>>>>>>>>>> replication controller
>>>>>>>>>>>>>>>>> php.php.domain to Kubernetes layer.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I couldn't investigate the agent side, since containers
>>>>>>>>>>>>>>>>> are destroyed.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Any idea?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> email : manu...@wso2.com / man...@apache.org
>>>>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>>
>>>>>>>>>>>> email : manu...@wso2.com / man...@apache.org
>>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Regards,
>>>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>>>> Software Engineer
>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>>
>>>>>>>>>>> email : manu...@wso2.com / man...@apache.org
>>>>>>>>>>> phone : +94 772492511
>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Rajkumar Rajaratnam
>>>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Regards,
>>>>>>>>> Manula Chathurika Thantriwatte
>>>>>>>>> Software Engineer
>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>
>>>>>>>>> email : manu...@wso2.com / man...@apache.org
>>>>>>>>> phone : +94 772492511
>>>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Rajkumar Rajaratnam
>>>>>>>> Software Engineer | WSO2, Inc.
>>>>>>>> Mobile +94777568639 | +94783498120
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Regards,
>>>>>>> Manula Chathurika Thantriwatte
>>>>>>> Software Engineer
>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>> lean . enterprise . middleware
>>>>>>>
>>>>>>> email : manu...@wso2.com / man...@apache.org
>>>>>>> phone : +94 772492511
>>>>>>> blog : http://manulachathurika.blogspot.com/
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards,
>>>>>>> Nirmal
>>>>>>>
>>>>>>> Nirmal Fernando.
>>>>>>> PPMC Member & Committer of Apache Stratos,
>>>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>>>
>>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Imesh Gunaratne
>>>>>>
>>>>>> Technical Lead, WSO2
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Raj
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Gayan Gunarathne
>>>> Technical Lead
>>>> WSO2 Inc. (http://wso2.com)
>>>> email : gay...@wso2.com | mobile : +94 766819985
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Nirmal
>>>
>>> Nirmal Fernando.
>>> PPMC Member & Committer of Apache Stratos,
>>> Senior Software Engineer, WSO2 Inc.
>>>
>>> Blog: http://nirmalfdo.blogspot.com/
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Raj
>
--
Raj
