Strikingly shared their Kylin on AWS use case on AWS summit 2018 Shanghai event. They will do the second sharing in this week's AWS summit Beijing event.
They run Kylin in docker and made a Kylin docker template for EMR (build dependencies in, update Hadoop conf on docker run). Besides, they developed a scheduler component which will coordinate the Kylin nodes so that they are decoupled from each other. The Kylin cluster can grow up and shrink on demand, serving their workloads from millions of users. Here are some references: https://www.slideshare.net/ChaseZhang3/deploying-apache-kylin-on-aws-and-designing-a-task-scheduler-for-it https://www.slideshare.net/ChaseZhang3/aws-summit-strikingly-analytics-103889571 Their docker template and scheduler are not open sourced yet. If the community has the need, I believe they are willing to share it. 2018-08-06 10:00 GMT+08:00 ShaoFeng Shi <[email protected]>: > Hi Sonny, > > EMR HBase read replica is a great feature, but we didn't try. Are you > going to using this feature? or just want to deploy Kylin as a cluster? > > If putting Kylin metadata to RDS, can it be easier for you? > > 2018-08-04 0:05 GMT+08:00 Sonny Heer <[email protected]>: > >> we'd like to use emr hbase read replicas if possible. We had some issues >> using this stragety since kylin requires write capability from all nodes >> (on login for example). >> >> idea is to cluster kylin using multiple EMRs on master node. If this >> isn't possible we may go with separate instance approach, but that is prone >> to errors as emr libs have to copied around.. >> >> ref: >> https://aws.amazon.com/blogs/big-data/setting-up-read-replic >> a-clusters-with-hbase-on-amazon-s3/ >> >> Anyone else have experience or can share their use case on emr? >> >> Thanks! >> >> On Thu, Aug 2, 2018 at 2:32 PM Sonny Heer <[email protected]> wrote: >> >>> Is it possible in the new version of kylin to have multiple EMR clusters >>> with Kylin installed on master node but talking to the same S3 location. >>> >>> e.g. one Write EMR cluster and one Read EMR cluster >>> >>> ? >>> >> > > > -- > Best regards, > > Shaofeng Shi 史少锋 > > -- Best regards, Shaofeng Shi 史少锋
