Hi everyone, I want to know, if Kylin could build cube olnly base on spark. I means that my environment doesn't support running MR jobs, and spark is running on Mesos.
In few days ago, I complied with "Fast Cubing on Spark in Apache Kylin" (http://kylin.apache.org/blog/2015/09/09/fast-cubing-on-spark/) and found that: only in the "#1 Step Name: Create Intermediate Flat Hive Table" shows kylin use Spark to create table, and the next steps are all using MR...... And I read the source code of Kylin,found that there is a file named with "engine-spark" while the other named "engine-mr" I guess that kylin could build cube only through Spark, however, I try my best to deploy the Kylin but there is nothing to do.... so I want to someone to answer my question, and it would be very kind of you to tell me how to deal with this problem Thank you for helping me.... ------------------------------------------------------translate------------------------------------------------------------------------------------- 大家好,我有一个问题想请教大家:我想知道 kylin 在build cube 这个过程中,是否能够完全运行在spark中,而不是依赖或则还需要运行在MR上. 因为在之前我按照官网上面所给出的"Fast Cubing on Spark in Apache Kylin" (http://kylin.apache.org/blog/2015/09/09/fast-cubing-on-spark/)这个配置示例进行了配置,但是发现,只有在 "#1 Step Name: Create Intermediate Flat Hive Table" 这个步骤中 是使用的spark ,后面的steps 都是使用的MR..并且我在kylin-1.5.3 的源码中发现了"engine-spark"这个文件夹,以及"engine-mr" 因此猜测是能够完全在build cube中运行在spark 上的,同时根据kylin 的更新日志上面写的,他们能够运行在spark上,只是速度要比MR 稍微慢点...我想请问大家,如果可以完全运行在spark上,那么我应该修改哪些配置?(我猜测kylin_job_conf.xml 这个可能要修改) 请大家帮助我 谢谢大家... -- View this message in context: http://apache-kylin.74782.x6.nabble.com/kylin-build-cube-based-on-spark-tp5447.html Sent from the Apache Kylin mailing list archive at Nabble.com.
