Further thinking on implementation for k8s, we need to improve in 3 places.
1.) Need to introduce min=0 for autoscaling policies
kubectl autoscale rc foo --min=0 --max=5 --inflight-request-count=80
2.) Have to config auto scaler for use load balancing factor
(inflight-request-count) - K8S have extension in Auto scaler
3.) Improve load balancer for hold first request (until service running)
On Tue, Oct 11, 2016 at 2:24 PM, Imesh Gunaratne <im...@wso2.com> wrote:
> On Mon, Oct 10, 2016 at 8:01 PM, Sanjeewa Malalgoda <sanje...@wso2.com>
> wrote:
>
>>
>> When we do container based deployment standard approach we discussed so
>> far was,
>>
>> - At the first request check the tenant and service from URL and do
>> lookup for running instances.
>> - If matching instance available route traffic to that.
>> - Else spawn new instance using template(or image). When we spawn
>> this new instance we need to let it know what is the current tenant and
>> data sources, configurations it should use.
>> - Then route requests to new node.
>> - After some idle time this instance may terminate.
>>
>> If we were to do this with a container cluster manager, I think we would
> need to implement a custom scheduler (an entity similar to HPA in K8S) to
> handle the orchestration process properly. Otherwise it would be difficult
> to use the built-in orchestration features such as auto-healing and
> autoscaling with this feature.
>
> By saying that this might be a feature which should be implemented at the
> container cluster manager.
>
> *Suggestion*
>> If we maintain hot pool(started and ready to serve requests) of servers
>> for each server type(API Gateway, Identity Server etc) then we can cutoff
>> server startup time + IaaS level spawn time from above process. Then when
>> requests comes to wso2.com tenants API Gateway we can pick instance from
>> gateway instance pool and set wso2.com tenant context and data source
>> using service call(assuming setting context and configurations is much
>> faster).
>>
>
> I think with this approach tenant isolation will become a problem. It
> would be ideal to use tenancy features at the container cluster manager
> level. For an example namespaces in K8S.
>
> Thanks
>
>>
>> *Implementation*
>> For this we need to implement some plug-in to instance spawn process.
>> Then instead of spawning new instance it will pick one instance from the
>> pool and configure it to behave as specific tenant.
>> For this each instance running in pool can open up port, so load balancer
>> or scaling component can call it and tell what is the tenant and
>> configurations.
>> Once it configured server close that configuration port and start traffic
>> serving.
>> After some idle time this instance may terminate.
>>
>> This approach will help us if we met following condition.
>> (Instance loading time + Server startup time + Server Lookup) *>*
>> (Server Lookup + Loading configuration and tenant of running server from
>> external call)
>>
>> Any thoughts on this?
>>
>> Thanks,
>> sanjeewa.
>> --
>>
>> *Sanjeewa Malalgoda*
>> WSO2 Inc.
>> Mobile : +94713068779
>>
>> <http://sanjeewamalalgoda.blogspot.com/>blog
>> :http://sanjeewamalalgoda.blogspot.com/
>> <http://sanjeewamalalgoda.blogspot.com/>
>>
>>
>>
>
>
> --
> *Imesh Gunaratne*
> Software Architect
> WSO2 Inc: http://wso2.com
> T: +94 11 214 5345 M: +94 77 374 2057
> W: https://medium.com/@imesh TW: @imesh
> lean. enterprise. middleware
>
>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692
Blog : http://lakmalsview.blogspot.com/
_______________________________________________
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
