On Tue, Oct 11, 2016 at 2:44 PM, Lakmal Warusawithana <lak...@wso2.com>
> Further thinking on implementation for k8s, we need to improve in 3 places.
> 1.) Need to introduce min=0 for autoscaling policies
> kubectl autoscale rc foo --min=0 --max=5 --inflight-request-count=80
> 2.) Have to config auto scaler for use load balancing factor
> (inflight-request-count) - K8S have extension in Auto scaler
We may have some logic similar to this.
required_instances = request_in_fly / number_of_max_requests_per_instance;
if (required_instances > current_active_instance)
//This need scale up
if(required_instances < max_allowed)
spwan_inatances( required_instances - current_active_instance );
//Cannot handle load
//This is scale down decision
if(required_instances > min_allowed)
terminate_inatances( current_active_instance - required_instances );
> 3.) Improve load balancer for hold first request (until service running)
> On Tue, Oct 11, 2016 at 2:24 PM, Imesh Gunaratne <im...@wso2.com> wrote:
>> On Mon, Oct 10, 2016 at 8:01 PM, Sanjeewa Malalgoda <sanje...@wso2.com>
>>> When we do container based deployment standard approach we discussed so
>>> far was,
>>> - At the first request check the tenant and service from URL and do
>>> lookup for running instances.
>>> - If matching instance available route traffic to that.
>>> - Else spawn new instance using template(or image). When we spawn
>>> this new instance we need to let it know what is the current tenant and
>>> data sources, configurations it should use.
>>> - Then route requests to new node.
>>> - After some idle time this instance may terminate.
>>> If we were to do this with a container cluster manager, I think we
>> would need to implement a custom scheduler (an entity similar to HPA in
>> K8S) to handle the orchestration process properly. Otherwise it would be
>> difficult to use the built-in orchestration features such as auto-healing
>> and autoscaling with this feature.
>> By saying that this might be a feature which should be implemented at the
>> container cluster manager.
>>> If we maintain hot pool(started and ready to serve requests) of servers
>>> for each server type(API Gateway, Identity Server etc) then we can cutoff
>>> server startup time + IaaS level spawn time from above process. Then when
>>> requests comes to wso2.com tenants API Gateway we can pick instance
>>> from gateway instance pool and set wso2.com tenant context and data
>>> source using service call(assuming setting context and configurations is
>>> much faster).
>> I think with this approach tenant isolation will become a problem. It
>> would be ideal to use tenancy features at the container cluster manager
>> level. For an example namespaces in K8S.
>>> For this we need to implement some plug-in to instance spawn process.
>>> Then instead of spawning new instance it will pick one instance from the
>>> pool and configure it to behave as specific tenant.
>>> For this each instance running in pool can open up port, so load
>>> balancer or scaling component can call it and tell what is the tenant and
>>> Once it configured server close that configuration port and start
>>> traffic serving.
>>> After some idle time this instance may terminate.
>>> This approach will help us if we met following condition.
>>> (Instance loading time + Server startup time + Server Lookup) *>*
>>> (Server Lookup + Loading configuration and tenant of running server from
>>> external call)
>>> Any thoughts on this?
>>> *Sanjeewa Malalgoda*
>>> WSO2 Inc.
>>> Mobile : +94713068779
>> *Imesh Gunaratne*
>> Software Architect
>> WSO2 Inc: http://wso2.com
>> T: +94 11 214 5345 M: +94 77 374 2057
>> W: https://medium.com/@imesh TW: @imesh
>> lean. enterprise. middleware
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
Mobile : +94713068779
Architecture mailing list