Hi All,
WDYT on this suggestion? Is it worth to do some feasibility analyze and see
is there any space to improve?
Even without this suggestion we need to think about how per tenant/service
instances spawn with the help of smart load balancer.
So its better to discuss that and finalize it. If we have done something
for app cloud may be we can have a look at on it first.
Thanks,
sanjeewa.
On Tue, Oct 11, 2016 at 3:07 PM, Sanjeewa Malalgoda <sanje...@wso2.com>
wrote:
>
>
> On Tue, Oct 11, 2016 at 2:44 PM, Lakmal Warusawithana <lak...@wso2.com>
> wrote:
>
>> Further thinking on implementation for k8s, we need to improve in 3
>> places.
>>
>> 1.) Need to introduce min=0 for autoscaling policies
>> kubectl autoscale rc foo --min=0 --max=5 --inflight-request-count=80
>>
>> 2.) Have to config auto scaler for use load balancing factor
>> (inflight-request-count) - K8S have extension in Auto scaler
>>
> We may have some logic similar to this.
>
> required_instances = request_in_fly / number_of_max_requests_per_
> instance;
> if (required_instances > current_active_instance)
> {
> //This need scale up
> if(required_instances < max_allowed)
> {
> spwan_inatances( required_instances - current_active_instance );
> wait_sometime_to_activate_instances();
> }
> else{
> //Cannot handle load
> }
> }
> else
> {
> //This is scale down decision
> if(required_instances > min_allowed)
> {
> terminate_inatances( current_active_instance - required_instances );
> wait_some_time_to_effect_termination();
> }
> }
>
>
> Thanks,
> sanjeewa.
>
>
>> 3.) Improve load balancer for hold first request (until service running)
>>
>> On Tue, Oct 11, 2016 at 2:24 PM, Imesh Gunaratne <im...@wso2.com> wrote:
>>
>>> On Mon, Oct 10, 2016 at 8:01 PM, Sanjeewa Malalgoda <sanje...@wso2.com>
>>> wrote:
>>>
>>>>
>>>> When we do container based deployment standard approach we discussed so
>>>> far was,
>>>>
>>>> - At the first request check the tenant and service from URL and do
>>>> lookup for running instances.
>>>> - If matching instance available route traffic to that.
>>>> - Else spawn new instance using template(or image). When we spawn
>>>> this new instance we need to let it know what is the current tenant and
>>>> data sources, configurations it should use.
>>>> - Then route requests to new node.
>>>> - After some idle time this instance may terminate.
>>>>
>>>> If we were to do this with a container cluster manager, I think we
>>> would need to implement a custom scheduler (an entity similar to HPA in
>>> K8S) to handle the orchestration process properly. Otherwise it would be
>>> difficult to use the built-in orchestration features such as auto-healing
>>> and autoscaling with this feature.
>>>
>>> By saying that this might be a feature which should be implemented at
>>> the container cluster manager.
>>>
>>> *Suggestion*
>>>> If we maintain hot pool(started and ready to serve requests) of servers
>>>> for each server type(API Gateway, Identity Server etc) then we can cutoff
>>>> server startup time + IaaS level spawn time from above process. Then when
>>>> requests comes to wso2.com tenants API Gateway we can pick instance
>>>> from gateway instance pool and set wso2.com tenant context and data
>>>> source using service call(assuming setting context and configurations is
>>>> much faster).
>>>>
>>>
>>> I think with this approach tenant isolation will become a problem. It
>>> would be ideal to use tenancy features at the container cluster manager
>>> level. For an example namespaces in K8S.
>>>
>>> Thanks
>>>
>>>>
>>>> *Implementation*
>>>> For this we need to implement some plug-in to instance spawn process.
>>>> Then instead of spawning new instance it will pick one instance from
>>>> the pool and configure it to behave as specific tenant.
>>>> For this each instance running in pool can open up port, so load
>>>> balancer or scaling component can call it and tell what is the tenant and
>>>> configurations.
>>>> Once it configured server close that configuration port and start
>>>> traffic serving.
>>>> After some idle time this instance may terminate.
>>>>
>>>> This approach will help us if we met following condition.
>>>> (Instance loading time + Server startup time + Server Lookup) *>*
>>>> (Server Lookup + Loading configuration and tenant of running server from
>>>> external call)
>>>>
>>>> Any thoughts on this?
>>>>
>>>> Thanks,
>>>> sanjeewa.
>>>> --
>>>>
>>>> *Sanjeewa Malalgoda*
>>>> WSO2 Inc.
>>>> Mobile : +94713068779
>>>>
>>>> <http://sanjeewamalalgoda.blogspot.com/>blog
>>>> :http://sanjeewamalalgoda.blogspot.com/
>>>> <http://sanjeewamalalgoda.blogspot.com/>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Imesh Gunaratne*
>>> Software Architect
>>> WSO2 Inc: http://wso2.com
>>> T: +94 11 214 5345 M: +94 77 374 2057
>>> W: https://medium.com/@imesh TW: @imesh
>>> lean. enterprise. middleware
>>>
>>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>
>
> --
>
> *Sanjeewa Malalgoda*
> WSO2 Inc.
> Mobile : +94713068779
>
> <http://sanjeewamalalgoda.blogspot.com/>blog :http://sanjeewamalalgoda.
> blogspot.com/ <http://sanjeewamalalgoda.blogspot.com/>
>
>
>
--
*Sanjeewa Malalgoda*
WSO2 Inc.
Mobile : +94713068779
<http://sanjeewamalalgoda.blogspot.com/>blog
:http://sanjeewamalalgoda.blogspot.com/
<http://sanjeewamalalgoda.blogspot.com/>
_______________________________________________
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
