What about Kubernetes? More heavy weight, but eBay overall is putting all its eggs into that basket :-) http://googlecloudplatform.blogspot.com/2015/07/Kubernetes-V1-Released.html
On 11/3/15, 5:26 AM, "Ted Dunning" <[email protected]> wrote: >You need to also decide whether Helix has momentum. The low traffic on the >mailing list may be because the project is mature, but may also mean small >user base. > >That is a very hard call to make. > > >On Tue, Nov 3, 2015 at 2:00 AM, Li Yang <[email protected]> wrote: > >> One important goal is the cluster management should be as simple as >> possible and introduces minimal dependency. This will allow Kylin to be >> self-contained, easy to deploy, and is able to run on different >>computing >> platform. Especially we don't want to be locked-in by any platform. >> >> Helix is good as it's just a lightweight 700 KB lib and the only >>dependency >> is zookeeper (which everyone depends on nowadays). >> >> YARN is great but implies Hadoop, meaning Kylin may have trouble >>running on >> Spark alone. >> >> Marathon and Mesos is super great but seems a overkill. All Kylin needs >>at >> the moment is leader selection and task dispatching. >> >> >> On Tue, Nov 3, 2015 at 3:03 PM, Ted Dunning <[email protected]> >>wrote: >> >> > True, but it is one of the few options for running outside of a >>cluster. >> > >> > >> > >> > On Mon, Nov 2, 2015 at 10:43 PM, 周千昊 <[email protected]> wrote: >> > >> > > Marathon need Mesos as extra dependency. For those who use hdp or >> > cloudera >> > > as sandbox for POC or some production env, Mesos does not seem to be >> the >> > > standard component. I think it might be a concern for the users. >> > > >> > > Luke Han <[email protected]>于2015年11月3日周二 下午1:31写道： >> > > >> > > > Very interesting one, Marathon can run on Mesos which also could >> manage >> > > > containers... >> > > > Worth to have a try... >> > > > >> > > > https://github.com/mesosphere/marathon >> > > > >> > > > >> > > > >> > > > >> > > > Best Regards! >> > > > --------------------- >> > > > >> > > > Luke Han >> > > > >> > > > On Tue, Nov 3, 2015 at 1:12 PM, Ted Dunning >><[email protected]> >> > > wrote: >> > > > >> > > > > Try Marathon then. >> > > > > >> > > > > >> > > > > >> > > > > On Mon, Nov 2, 2015 at 5:06 PM, Luke Han <[email protected]> >> wrote: >> > > > > >> > > > > > Run on YARN requires to run kylin on "one" Hadoop cluster. >> > > > > > >> > > > > > Based on today's practices, Kylin's services always sit beside >> > > clusters >> > > > > not >> > > > > > inside to have capability to interactive with more than 2 >> clusters >> > > with >> > > > > > just one Kylin deployment. >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > Best Regards! >> > > > > > --------------------- >> > > > > > >> > > > > > Luke Han >> > > > > > >> > > > > > On Tue, Nov 3, 2015 at 12:12 AM, Henry Saputra < >> > > > [email protected]> >> > > > > > wrote: >> > > > > > >> > > > > > > Why dont we just leverage Hadoop YARN? >> > > > > > > >> > > > > > > We could use Apache Twill [1] to wrap it as YARN >>application. >> > > > > > > >> > > > > > > Helix is great but one of the limitations it could not >> provision >> > > new >> > > > > > > node when not already available. >> > > > > > > >> > > > > > > [1] http://twill.incubator.apache.org/ >> > > > > > > >> > > > > > > On Sun, Nov 1, 2015 at 7:01 PM, 周千昊 <[email protected]> >>wrote: >> > > > > > > > Hi, all >> > > > > > > > Kylin currently manages the cluster itself, so far it >> > works >> > > > fine >> > > > > > > since >> > > > > > > > there are only 2 roles for the cluster, job engine and >>query >> > > > engine. >> > > > > > > > In order to reduce the complexity of deployment and >> > support >> > > > the >> > > > > > > coming >> > > > > > > > streaming feature, I suggest kylin have a better cluster >> > > > management. >> > > > > > > > There are several candidates I think can be >>leveraged. >> > > Apache >> > > > > > Helix >> > > > > > > > seem suitable for Kylin in my opinion. >> > > > > > > > I would love to start a discussion about cluster >>manager >> > of >> > > > > Kylin, >> > > > > > > how >> > > > > > > > we can do it the better way, and which component we should >> > > choose. >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >>
