Why dont we just leverage Hadoop YARN? We could use Apache Twill [1] to wrap it as YARN application.
Helix is great but one of the limitations it could not provision new node when not already available. [1] http://twill.incubator.apache.org/ On Sun, Nov 1, 2015 at 7:01 PM, 周千昊 <[email protected]> wrote: > Hi, all > Kylin currently manages the cluster itself, so far it works fine since > there are only 2 roles for the cluster, job engine and query engine. > In order to reduce the complexity of deployment and support the coming > streaming feature, I suggest kylin have a better cluster management. > There are several candidates I think can be leveraged. Apache Helix > seem suitable for Kylin in my opinion. > I would love to start a discussion about cluster manager of Kylin, how > we can do it the better way, and which component we should choose.
